“Calhoun 


Institutional Archive of the Naval Postgraduate School 





Calhoun: The NPS Institutional Archive 
DSpace Repository 


Theses and Dissertations 1. Thesis and Dissertation Collection, all items 


1988-12 


A framework for software development. 


Hughlett, Eric C. 


http://ndl.handle.net/10945/19350 


This publication is a work of the U.S. Government as defined in Title 17, United 
States Code, Section 101. Copyright protection is not available for this work in the 
United States. 


Downloaded from NPS Archive: Calhoun 


| Calhoun is the Naval Postgraduate School's public access digital repository for 
F (8 D U DLEY research materials and institutional publications created by the NPS community. 
«ist | | Calhoun is named for Professor of Mathematics Guy K. Calhoun, NPS'‘s first 


NY KNOX appointed — and published -- scholarly author. 

4 | a | 

“ea LIBRARY Dudley Knox Library / Naval Postgraduate School 
http://www.nps.edu/library 






411 Dyer Road / 1 University Circle 
Monterey, California USA 93943 


tte = or 8 hlUas lA ll eee SS _- > ee ss 
rrr ee eee ee oe a et eg ee ey eee ee 
; s FoF CAO roe 6H yO Pothe F-test Dba tO PhEN8 he Heke PRR gts? 0 2S ere ee = ae ONS OS Ree wate 
Tefal te: perme peor ae, ee Pr pyeA ea ae eo vneamunchaataet Magik oh Dia RRM dBi d othe NEE in teebentne nf TAS pe | Aire oe ea an peti Cg Ie, idl TEST Pah oP EPCORT LT OT ahs POPES B8 GNA ae praren hea tatoo, SP AOE alla et gM Mite ce Flale pa Alpes i. Ratha tee AUST eo 
PF ORD eT ah BP ANON 6 Da yee Ce cokes OTe rare ey heer Ye ee fe a ee ne arial « Cpa Pee ipa FE Aah FUEL BLE o ete MEHR 0 FTE SA te Ho AA FOIA tala tPF ILM an east s NMA, AP AO finn key at, ee Widath ny tele Sa qs ee teetin oy 7 belts ic Tabata aay Analy hr 6 ular Gr tbechehs Wy Ne mo V/ Gee Bot 0 Fy BW /egite te 
: % Soe Fe Op te 5 Plea Pree epapetatenteg ie | BA ray wicenp pear taae Me pe ATT EL NP er es san tides arate Bil AE at © 0 Oren sige on pete ee tearm marr Ate te ach ace bs koko et eu Saath eit Coco Lo Ree Ma > fpr Bete atee ta aa Stone? 
. ; baat Mo iPr 8 Beis natntet: Je Ian . F : +t eet gins . ; ~ ' te - e fe aso 
A, t FY APE De papretny AAD Ie boda tne Ke Fare. f+ wo fO.r0 YOM’ %- Polin'te esos on A afertpbs eee ahaaeeanse Fm Pets PQy ea herd SL 6025 AWA Eg CA ISS A bee 8 EA OAc at BS aes em Leth Tr ke DA Ter he Mhe Wh ae: MMe Awe oredr ee bs a Se ha Leen tay cma lies has 
ae as ¢ “Hehe Bea LMCI RINE Rammer) UR. WAL athe ARE Set 2a) ORATION GES Spt etn Mah F rent am BA Petre Suites hAehWOek- ASN ENE Ra Pore tee ree tar ed 
4 " hoe . » Sata are Qa bt Mopar x Fay Rape eaten AGS 19a, Ip: bre oper on ameter 
Ww |. AnaeWteth on! is athe” heron Aygo” 

= ' 3 








2 PP Ae gh abel Rech aD 0 att A 8 BLS en NIE A OM ent 0 RT Lay hes Suk af A Difco COANE Bab onl: # FY : he ALN yoirvitg 
Lane «8 of. Mase ra: arene fat. eT ee a DAMIER IE oO POM Dely art Ref, ~ = an herent Aaa Af eae Ad ho @stend Noe 
Se ear carerre Big enna s fot Al Bee Pacha Seal da F RP 9b ATR RAMOAT HEA, &, UNE an 4 tot Ine tipetks Renew te eke mhetin Peae Aatenchre ins pon Fuba met ire espe ed ja State Relete ei nates Abram eat MAG Heh by GU ASihe trae ee 
Fido 3 se depres aia ¥e Pos-oni be Noms Se PoE taawr, ne fa Reet Ake ety bat + i ee ental af LS Pats h. eee ems ee eae sae een a rater Heer t-arod ap ee onare aS ae earn te ws Qh Dog ts Ute. QAwD &, ere Re, a Sapa re 
: = 4 gop Att, ~ eo t ee £ ee : r a - : : S 
me Pate h OD cK DAA oh Seed EAMG OA A aDiad tet AO IFA Rec Bb at aK Mod wil pepe er pene erry sheet y BAAD Fae 08s SME 0 AF ear eR ohS hay, 90 9 LDR ATe PDA Claes Me FI MO Mat as peoet " Ato e i bg pawn yaa x ee an Se - ate leg eae ue oe arg nee 3 
ate 7 eran re . : : eA ap eye es he rer 8) Tarts Aetna tah Nanas wae MODAL Ue He HNO AS Re bre 
Pea Rte Ae BoTE Ap yO BLADE (DO ma pte fey a ek mapa athe Brite Rethel 
hs Ronee Rasa Rita My Bo tay —ppeaphe rita 
” toe Rog By Sire tng Ose eds, 
: sn Ihe Ao tne beh annette He BERS S, Sh a a 
i) 





¥ es ‘ A 
ra AES AIS aS taf reset Bred RC Hak mit bh tek sok. PF arc Galle winn ee 2 


Ook AP. 2 BaP oO BM Ae REL LM MEE Le a tet We = of eho AG APF aU REVEL woe $28 hee 6 FMEA oh b08 fh Bue Me ; 
, he sm ; = ™ aie f Am PA het rN ad Blethen? . ’ 
Det line ab degonecn cannes oom anna Seo bret er NmRd A peclaia ablokerticpanmbeAnd* haf th dps-sbeae FE a teuncenaqurtenripy Nee Ed chor Rote eae sts Seen TORS, Caer ret eG OW NHS NOSSO, X : hE EO 
Ae aims aces ee etna iaa rte sbeet apes ab! oy phn te ano PPT DADA LO aN Nook wil OW AMEDD GS of Hake OI PrN toed F are FS Pblarly Man! Um od Pint hatin Rene. APetesna eC ane. Cie a RP A np) econ ene vo Fe Sous boleh Thug casas ngraanhy hc enh! SI Aether A ahsa heir oeah by Ayenate Riven i Pearyi Aate are tieerere er yy arty ete 
ant Wat od tdichegedntnt tnstt pcota cede naet ant ranted iin tanatoiiasact pate Ge Gob perreripe Lens rye rele Lonererarenne se an aspire ale pale iP bk eae fs. 3 me. tet ee Pampa et poet Sasaees eet een te > Eaten yieran £979 petctery eek’, Sek ab meee oe Rely eR Aap m Me bone ate Be Re Dadlinrta-lnr thidin-tartactier 
mage " eee apa Ale Ss = PMT aD NYS Cole dea Ba Pale lA ATER POM Mel Fah Fa hal EK Eet ie ek He Ae Semen reese May eh Te Pgh Ra Ra! Si ne ne cout ' gers fe pie, ee sa A en te adnate = “howe 
. le PN = De EO Rh lhl A, it POT ot WE ALLY 0b D OAR 8 Ae Fin Gd BAUME PELE RL ev @ $e Bah Arf tritc Moiet, 9 oak pe tkoas teak ere 6 easy Pe Stamnes = AePteme se ‘ i pete: b P4 Belper: ; Lo sh arse mites agen ce —— ai lear pre a Metaite \ “a nS a na! eich L Seatetibaherthe  wasnet he 
. : a “ . BALM ete gt NARs tie AeA) cl eta” Lagtla tek og Macy Altace NA oan Rand ALB LL Me OY AY UPA ba eS Merino On bev ten hig 
F 1‘ oie Mn | 
ft “ 





J tut? UF f 4 
ie ‘ ‘. o - 
chin bette aS ND OEE RO pe dd mM PE 0 robe aE TY De eerie gf tsk see Aree? fat pl eh RFR DM ED, BH 3 4 . uw y ee ie 
. ‘ 2 ' IF aTh eG ant: AAA AN Rtea eye te Mette Mar Vat Fh . 
ree os te espe ilies ce o_o rte eke pry Seigets pauccetoeneen ete aanent tr ANT aet eR nak enh Ot ny Tan ADP este Retind NLT Nate nde SINT tes ban R gms APE Ls § sore ned Oa een 
ab dtd iad cata ana amadeied ; sr Se en can RR ch Lhe task Sea ae Fig eben Stn hao hah Peer Auer suena Mba Anailon, Be VIAN AB oe! ele e9, Seed ite tar bre ~ . . Pose Teac dina s By Redeem 
saat ab ware tahe ? Ke PPh Sad A whe rte hath aR FA athn ti OME CU Feit Mhe PEM Mos Fr VERDE 0 BALK bee g hohe PD Pockichoatrctic taht he Sse Us thet APRA, ate soauatane be mt ouch aos ra % ret Ted ted poe Sones pale Met ieee me NS Penta eee » Sra ged Note toe a hetega ov omnneneme en marae sete ea Me TUNG A et ieee ey =e Maes De Mae ND- 
4 = . : Pp : alee, Kati hates ee neh CP Be MAY Rect oroad MMe GA batt Shel “ _ OF ee ail abl e' n . eae w = ware aes ‘ sup A 3 3 zs 
. , } 2 RA hapa Peg 5 2 ; aA Mele re Rete nasa et a com ata NT em TM Me NRT) oy His Bd 9 dates he Bere AIP IE R ste ene itso Se Ne 
\ x ’ rgrey ie end aie 7 Sap her % : Sa Nhe ee. aretha Tepe MA Mein te tm, Sit stern FAI ON he The SMe eaoiig-t Se ar la en 





atin icsaih tatteeds Aaland a okotae tail ’ 
mam hi a ath a> OE GO AE OK SLM ALM Spe A rE? adr ot MAE ne O Rds Kat BLAS clr OB MRO NGM, PRAT OA A POD y LMI OR 2 
f le OAT a cee flak Aka De Bots AEG RAY AAA a 2D WP sae SUE EIN Daf attr Rd Reee LI Yo WM : : P fs 
RIAN AE A DI Reha he here stden Aree CE Warped VA tpl at ae } 
“ . ey : MYA aay tah epee Martyn Mee aah, Bs ha eked Rye iIhy tos er = DB AM, eI e Rie le TAG HIN Tin OND Mt § 
: F eee Certara tN LER Tee Mee tne ee Pech arlene! An Maen Setar BoM, ate Mayes Hd MSM Mote 
Fig: Romaine Mes Menten aa dole 8.5 & ele hy" — woh or ay bye pe 
< De ey Qe btelg Raat. ay te 5 Rata Latins SY > 
Rack tee eR Y SES t Teg Rag RV ite Pd Gath ee Sy eres tuhoseare 
ae atc “inher Wiebe ahem Aa gta 8S Gs ODP APPR gra? Me ET an Hit As Uae owner 
ee ee Al a NiO gna eecare 











Spor elt Pale Thay antl: gation et pa eae Ma ta teaser oe art 
CR Want sy Malley nar Pe ae aay, at My TL WS ia ea heath ae? Aa Noo o Nv af Tage Mm Hea AON Paeheiege Pee rae aye roles 
tanh Detach SEUN hemlet Raab CFM ag Mk Alpe tes arn CS Masog Tete a, Ne Meet Ahan WHT 
Fle Meee g Veockath -AFARor ey Tp Nave eat a Sart ae ii Ara aa teat VO be Poti hp the ys "ho-tn O.eNe tint Part ay aw Ne hp Bang ee et 
ola ite bony IER Sah Yat Rete SoA R BD shy MS Te My (hs Ree Olen Maree 4 
ef Rak, Rg Rhos Newnan, fil, TP ER ALGA OP Ry ent 2D le er The Hota ST PRS bay har Be Tmt em Dey Ba 
Penge el susan las ose et eA ” matory Ring Wyle Dsiterbe belo Sh Ar Ar area Sanne Sgunvensnemautate es ie Ciphrme 
we & pebeiemill te OST elena tet 2 POLE PAS A70D Oi kineet RI PTF ) ape Bir é 7 A os nop Om AS Reh Aare weg tes ein, AGTH tnt Mein Boga My Rare ara 9 ay erin 7s Rito» de Got te 
0 ei teria’. _—" > ie ; « ; ET oS aM fete Mit Dale Pa tam, we ko nly me n * r 2 S mf 
iF teak meena emake erage tae eM Caf Sates Pa ka tO oa Ge aes Se Bana n OHA feet LON et ape gat ae ek a aga Dg AR a NN ae Aion soe eee Tiverton arcane eae ee aN eae a nat a Reeth an CES 
OG A th A Be an 2 th ton? oh AF Es OPM DACA in RAM, aig ge” r Ae rt i errhaarves 9B Metres & tl. dled Bh I GM Aghia hee LOD Le ee de ee es ST re) a edie Ble Breer haggs A Oe she) ® tang “A —~ . “ — < m rape) a tr lp EEA een eae ctinchgse tomo EIA AN I Ps bg Phy See 
aa robe x ed Se” orp nal Sets 7 Ste 5 ae z ; s " en ee arp me ASR Maedendes We * Pretend AU Ae Dee ae te Wein! ee Se Dang tacln. Tae fe Hee ek Term Me am ; 
st ghana eEGtematar os aamerienr willie, | nein oe Sen Candas cao, eae oe ocean ee agen oe Se hecadangites Akl Molton tet AGOO~S. Naftntsiahdt YD MAein Pino} onmrelnsty ace segann eer Seen SSS roars Rt ALY eee Mee ae eee och Deke te tebe a Anti Facelac Nerd te Ses tare Soe tal ete ag Mth Hud: Acetate Rs ba nan arse AYES NS Ne Aacfle any Ree Behe 
1 FG DORA AA, Le Relic Ph st aa, eicll: 1 ah sth tin tga nn P otelt ra ae oh nella tie 98 ets Re LO PRANL Mab tier tytn a Arend OFMSY mebagent aAhade dS Lattin ity dete tte on BODE WG AAPL MPO He Sor MBAS IRE My pat at. SalI adapt ML? nat Spain rey baled cost mh] Des tate: opt hh Neale “ie Baro er re OP Otart Aedes ce. Worle & ete EA SN ly Ra tyv ty lv Rate hart 
ng fo Snare Lak Wes Bath sal ste Rennie te Rot ‘ar . my a a Piney Garry ee we ee Ty Red et Sa. oe dn Aah 3) di oP w pee . t ran lhl peal aa re Aares 
Senet Map ii mca e enone pes abo waren ahe epee ce immer sereyt ack yarns tinal Fo Pa mri En EA RS te nn Oe a RN sate ee Oe TN, ee ne  ameme trees Co ea arares rarer erage een arer emer ene ee eae 
a Seen paral ncaa ale ent, Pal ad Fs, leat ee nd * ae ie Re a eel “we weap pel ag -erp- 4 ; Nema tirtner e-perttr a = : 
Se a i et cer ee Sent SORE TLS AIRED OE ere Ope Re PLY OTT AIT ET np A MEEINY ReCtt Eetrcr a Oe acne et ree OE NOt Lae gaa es eg ere pee eee CN Sere pee Caen atten ale Sette ee 
. cpg a rt a ee ee ee ee ee are aaa dliemth pi pri 4 s : he dufisasufem ; ‘at F ip Phei hah PATIL Trt CLAM Ms — 1 eA Aen S : wy = i tenthrtde eam fe . n 
eo ae eee Comprar ws AAR wry etd. oRtih home A AIP D Fa B Nd apne Sere Ne oe Pa a oe ae ers 5 es Ee uheae eo apr Print rare went = om wet SAIN AA Pad Stebel My d aeeese rele NSS. Csi mrtg arr 4 Sse Peres pte enon eee Natal Rebtel a Coes atti tA in WADE er eee Teale 
Atha! “Accra ea staceece alt : peering seth amat—— | IS abot abet QOS PRED FO Pama ita Daly Met 82D” woth tan Por f obetintin ds, © Ci doin Be penne Nan te Derin tn O ee Mogren 9as dre sl taaSinhe nL 7 acest Lae A ond A Sa Ph. BONA PUN wah AIM gE hey A PDI Tale A hapten baat 4h. ete ott b ee ee ap lh Nese rer Srn Sota Cater nner Be Sete REN RS Ne eee 
ema a ch liaceeas 4 uead mpbrde porn PEF IE AO ohn MAS mo Pk hein aA tcl WN aed KEG  PemReD Lemons ot BB PB Pesigets Pudl A Mitel tal A ean nk Ps Deane ke Am oe | WD alah ade G7 8s lee PaO ReH Med aera Peitins eae hehe weak pant he RSE Dy ly Dakshin Sie? WHEE UR Aosae INDIES Se fe. Badin SpE WEN NIE To Weve Se yade helt sence err See oe eee Lest aiea igedigel ee 
bay alate Sete ee era eres ee tet PWT, yatteR lad as cemtind Wing ok a FAR aBet ition MILA DT wp Bes Be Ch tiga are peee yer oer 4 a en tart Sy EPR. 1 Nee ee eect eLearn ta ment oan tact d a be As & PAE Se ND Belen te Meenas Goel to Ney Vateahe ser Sgt Nae saan Ami tarsnahabe nny eget nnangana te Wetutyieetas bene N SS 
hatte sechgsin apivapkchscshin rer ler. a0 RM aR anie uaF OG eet oiayend BeO7Y cab aediedh bak detbedco eed eee es ana ieee ananaar ayant odes Bn dog Mlmtutatha bran. tle ae Ace malta MAES, Se eet gmAmainre Ke es eT eS eel . = Ry WAIN IRE Ya gh PUP IO MO PR ERED Ae belie po oe eR Pah eM BP Dan Aan nies ems tinge emai he Se te ata ei el tartar Or AO athe ty OA Oe 
aes sotpetiglt ean ATED Fob ig ea hgS i Prars ca” OE ANN ee Tat na TB SOFA LA on Soph pit da dss be satan Ae Be wt Pata tate! gel, eae ot -s AR neh ne see Fees 1 INE SOR tra gets Rt 89 pyre Dense RA An chee nen Ant ta “ aatinttye pte Met Pa. Ge Ls te eRy tary <b bette’ Rete Largs wav ta te tates Yelp Opens omnes SaMan Aime a egy BoA 2 cap’ erties ean MST ge ih ne Ane RA SM 
ie le oe ryirspre drgaain phic fates ethene WE faa westend afrePas” OIF Anbahel winpS Belin” coed buh ALS ~ Pereryrry rr pote Fans rmeraeppseiiel ae lage a . Nise tosiy al Se gy ea > BBs yang, et ered latin, HA SVAL hss Aiea Pectin Dihtes mPa tne steer Pee by eee eRe ao Nd ReaD Seta BED DREIL: hy Wh arte ME PM Perea as nal Mei eae Men ay elon ye NW NE 
Le te ee aw Raat Bet yo es : : ao é 5 . - hm ; . , A re re i Bae - , 7 ’ 4 2 Fe el Men Pg DM aon RN BEE OAC IRE GI bn Pe Eye pe eg tha ane Qh arabe 
gO ent peg: ten wCrren a8 oI pe surerioahed ts Ee Face YMA e del koa MAOH RE aR Cyc Anlagt a atin U ale FA Mellainiecath” - Nima UR Tae ft prarefiarst BA ne ‘4 at. teh det oy a Oe dae Ste Pam rating ate wrk Ratgsase hee ie Wren OSIRIS = Sea a abere SROSR, Life Peat ret % ar ep IE EAC Bo OHO Aa OAL ha ate try teetorip trae 
SEs ee SO reas 2 Accutrelidtetats 7 ifb bene t- Ses, A78-MANEN Ra Deed Pak m haf Bee ~ Ted nee Tent cee Pe fedannls Nenu pad et wre Stiinta “Reel Sins Mina AR eg aE E> SOA AA netowie see Artaonty ted aac dtr hNN, N Sn qteea ao death >-enartee Srhbr mantras wre oe e ears ston Spee 
taherk Sateaed ew. pz, : ee ot sh 28 ether Md a th en an Me Malad eB? aye Saal AO ae ite, Qe: thle PP Oe Ragerne  a~ee QE . 
hE el oh PRR Statens hint tT RON ODE LE phat Dc n Rall. Pathe et OPS Gra8 hl ah opener eben net at partes 9 Ee Rusora tare nado oe go Feri Pate 19 Sacto wre marae Dngtne FT, ws Alle adders een Bs Te ae ema BFE OSI Se anaattnong, 10 Aare etn et NAS, be Seek ARE Se en Nefae ts Lane tale Bite Saati iy 7-3 Rey a ee tee eres went 
siesta ahaad We Laval 8 tated eh PaO EAT hak ER Late aaah OVE AL eclialiett: | DA atin Fete Sen re See, ee Ae ee LPL finn Pr i nel ate 1, Pah Smtr Somes MOA LOT PEELE IS pay horet X. aE bh andy to® Bet Aanendeep doa Rsharien ag AES Me MOD Tw Sattar te tenancies Soir tate wats bts NOY ANS 
pet aa Cem Re DE fr Bo mle Male! eto LP Pn? te! Pe ak wet inde tt pat ord iatent res ras bdan Aare eae ee Poy — cans eri robert YS A retl tledigt as San fe, re ere Popa telt atet Note a imlea tng A IREEER A Ath Sapte Reeth enally! Rpt hn hb Baath enters obrk Me hee Phasptyy Mag Migs Aes . Pe ee crt ing al Mey tp Reig feet A eM pe eget ml Pa TD Maly MeO 
ened aM Flin Ra et tre hy LA PEP Daten Sth aan ic Dl hh VAO Bos Att at NEP GR gts me xh CMOS oF od —oeee mt negpelens naa ape fo Petes fet re ant a “ x Pree CPE peepee thn = CaR OS As ARATE oy afer Hr bem RAPE PE bal ATAPI eG BE Bee, ARG i DePgpnnctagd 19 Sea SaeoN 1 aro Austere. is eple BAe Ba wertewarth yah Nhs Ss tat he Tee et UPTON em PS ay Bs Ms De Nn 
ee ee eee ee ee ee ee ee ee a ee a ye anntteds pmrteeno Su ages ie = tall er ery moral Rin ae ry, ie ee Tal a? agin eye hee P bis PD LEN Te Me oe A mag ttn thes enti Maps Aw aaa Airtel eats ye too tiote OAD sheet sath see omaghh Re hel Oty Her OTe Lets Re MPR 8p ty temattea A -Pion Pat ba Ene bo 
Spanapeterdinesat ae tedene-aerxsecate aal-eai-gatandee?- Antennae eae toa IR Hi, WE. Ape PPR nage aOR TER A eigenamctad & elt Mahe rs = =e ire Nate mer tS ap abate Nera s Wipe <b eae <A Yr Ven oN thy eae UP Seer en tupeeenecnn ltac talc BRA taten an: tn 9 beta eh Sir hrnathe Wn Bra taSen a me Beinn he 
Sod wap in Seater tan iataed toni iaanon a ee get A ee ee Serene kOe F pn Wittrt neta had MT Dated to ek rea hon tam bat tet Nanay ASA Nan ns ery atten Sete efeeRiR obs KAS ome, MR eats aarenceertnctn tease Ween Dhaest ree mah: 
hajen wean het Rey Me Re 2 ie te net all, 0s ant obtain PIR Fim eth ta LO Balt tngtinntin ie welt @PllaS MP troaend. ~_— ee a = — o- =e esi ‘Coe p Fs wus mee Othe Rin ty Ral Mate ay Mu NO ee a astebe AEM FUE Aiea Wh be Roe Lyerly ee Fe ean ag: Ne Teed Ker OA Ta On pew BN tan aos 
FGA aa IS eae plata atc ay eS ate kabie te ore line wad ALPES hE (oS grin dentee hie —— mre wi aol Sai ome al | ~~ ed wet eas + fu 0 9 FSR et be clad te Sinn puiinartescal na. Manne he Bel Oy rein On Nasty S De Nak Madina amie De eh pith Rete er tate, Bi es Doe he Shee Os RN Ns FeO Aly Seta att tee Maar mo May hp Nagr ther 5% ag tt 
soaialh diate nik tales ath ot a ativan inet efi Linew ein! — Anan 0 leah a taal glh Dell wines eaten Bell t ENN, all nr ae ee a ee ar ton ae ee BP cn . pee mee POO Or nn een nn inp Grade Pye peat Sect QO eal oR TC: MMe Bia hy Saat Nar ley Te ar tg Mia oP, Dearsante tn ev -Aeslgnras martes me 
2 aoe ee eee ® aoe et Fret FOB re canard! wane BP Aig Pesto tai? ¢ tn taKednl tndina® nity - e sins = ¢ = ne kana Paes Pere fate PPR Sed ae tte RATT LOAD ty Mie they S Athan” SRA aD  <eR Tae aay «eagle nets OSes heat ere N estes te tld te tena naren gine paca sp- Shenae apie pm apr hietie A Sw ates sea eeeneeentie See 
ec pease ae! at ath Pal peat iMedia Bel ntl hy BO OF alent npenPalsdi2 Datighig 6 Ln ttasthandd $0 ths the ruthct eAetnabletin Dah Rnaeatbehan Sit bP BOS! Af ees se pee. " + gen Agta Rive ek Reins trent Ante e Sant thee noe Kiel «co baglnatte es Eilts Nala. ates, Pm nt - 2 Rasiet oete tte POSIT a ahaha Gooey ata eee Sr OP GE De emt tote he Ry, Date as 
tebe 6-0 eateetieee er aie alate alg act Aelia DBAS ap Par gti, ARE, gO hgh rt one whine Papier < otal soqelh adipic natalia dia Sans oho oe amare Be swlaas tela 1.05 setint tamer o git map aa spp rs ak cinta eel Seeman ating AR ity Seem gh RPO iy hcl EN TA rr AA OPE Ae Deeg: Pan tte sg AANA PD ae ee her Li Dan Reese Podhin e, Ty Drprtetnamrray ey thle tg Ae eoley FH seem, ee Behe cmp we tp Reh: As Te Me 
eat aA a Meee” ot agg? DIP tym NP oh sell tone Oli B28 HM ad Lian Lom Bntie® Bp tree! bial ign Sabet oF Pea PE a rte i onc Tag — eee ot RS al - fon ine Nxt aa Pree CBO) nip tnsiong > ad Cu ¥ Oy ado rtonena Xn. “Awtato neti te 1 ARE metas tot yomin hp bene Siren bey eee Hanes Bs Sov alla Ai Says tne" AS Rs Ney Tr Dis PeheerAa  Polonead 
Rte PM eel plas a mais M eaten eo Liel Gry Lad) Mallia ea oie me we wo el Anna pana arte AA Peraintnte or = Unde, 9 FAA ate lO WT prised ee epee re arr ae ee eae pbk aN NN td Mes OLED Sel lores Sy WPM Me eran aN Care P PA Dag activa Ney Aoniny an N Utara. Tot Ay des Pa Ta, Sora bina teers marae ee mati ete 
Sabena ier bie ckchndgn eigenen aeete pee ah? Po ma mbenne tenet a oon Fee Be® aiPalindinedd poaeltier so ra peels a es trea laae Abhay ~peritiaree ya 8 «Clie cn, cen rt fA tT At Ape Aspe SPS: Notte te Negi eA ots rin? Mah Prue "oc engin a he SH. Rea arte ett 
Sunn Fane Ved ak oh = SNE ON gs NIE Om caw tathath.ioMnr! ob Date Peat aad Baten ttes gh tant P0 thm Pal Be, meth - a pee —_ a Nee ne io eee a ae ROT OF ¥ i manera Ale tae eet eres eR Rte We etna mee Ra Moe? Aa AARIE, Dy = Mera BRS Ae te A sigs ben Rane: GA Neg Setaetl aD es unlaleg aay Man eM Yale inytepie ter awe 
we te” athe Setar Atha ate alamo bnr 2 nttatud r¥ Met eS EP PR i 5 - aoe: Pint #: “oq udetiaeehe RAR Seige SN? = “Nee a * =e are pe ee Yee Pena Ne 0 ah lion Mngt MeL Sect eter KN Ae ema Sahay My ATNeNE N ake ne tem. OP Apt ak Minlingi hs Re =e Nts Tp cd mn Re er Me ARMac ining cw Ae Moulin WEY 
. rire Cee ted wee te ee Lat Pei ee eatedinns bm gg bee aie te IA IMO PCE ATR i Nol aad: - rs wig! enon ae # Lf ~ Pn aoe a Prater tye Pata Mal efile relied apace ee tienen © Atte bt otter Wetdnn ing — nfca® KORE vacate Se ttt Renta anlhe: ye Oana Rey tee Ries ate hes 
mane Aree 9 cai phirindi in fn Sit ey at anise =" need Ban erate a Sk Se tae sta pp Pa wt? € hice Pa ; ~- te clipe 9 Meee, Nt Pa Nene Be Om see peoraey Nts ts Meech ERM, 6 AF sn tenmae ys GouTest fs Or om Meter On ae ey Wats tented Bye vp aero tate tage NA 
Pe ARE MO he AOA. TARE ge” FAW ART Pe tat oOo tang Pete mM PSUR Budhon a es Mf wel res ad ew fat iz ae oa “a oer on ty aid FLA, Aunaren cen tpt ete EMR AMET LMR AOL Ay Ta te'anr aa teh? Mote be tetera’ HAR OMe, & MEN cthee S. aTaplin Boerne stele Pela Saale PATE Pp Nos ew a li a pe aaa 
elaine seated Ant iata wt rd meen “oft - - ~ Pehatyowretia = meet Be — Pos ree ta Ba A ey Safe ee ws = a sett — oten pe a Se tecuet So tee Le An Getta! ten Rasp ten etian memes an PR OL Pie May pi apa Ne NTN PSS Se yer”. iene em 
aga, Tt de amdite akan at ce atta g 0 rte ita dies fucmdckee nite oe - L! Syeale es a os ee 4 wae Neat greasy Match e NAN e Agtefh~ He Athenee iehamda Wr ciae eebh hiya mere ange ath Ae hg a a Tag Re iy Mey Needy eA ah tnadparetiee 
oe eats sat scaate = el ettene MIe aot Metal et at: ooo au I BAN, 0S aheen lel ek oF Fn te PF n aM eMatng on — rie . ‘a if Pee ad a nee s vd =) ae! Poet L Pe aaa — aN a Ds oe 8 ee ee ee ee a ee ae RS k Phra et Tar he Me BT Se eer Ed mene tagitar te eat mpthiean ean te idling te ye Neat ® adn elon = beeps 
af pte Oe et A 0 ann” ait a eg re Nate tee veh gmetehy  wbeh es Say > sare J le tate ps a ° Cat atactariernn tm 7% Fo >» o Hiway” ae — na fe eee wee re Precio ellta © Be Math ti eR ON POA Nap Demy HN ye T et ailly R> Preethge See RAAAP Ele Oe PUM Be Peet fe Cele ee Re say Rote Leena athe ap & PHA The Chee Ay 
ee ot eel iT eetn tn Snr eaat gf Bata hob re? —< iA “epee = eee a ne a! . re “At ae araodts 3s pons, of fer F Of Se ee Pe e gett Sa ie Sean saath ere ee FR. Pa Hh ars puch mln, Ap te Ae. Minenegy” Mn oan pesball Lethe Raton ar erere > tnt Salar ane mote wan 8 Ta Pg ct ght Reale natin Pcs OO Nghe Manta Malpas 9S cay Meg th Tee eon Nea 
me, aap nO eR Datel? thas i ofa t ahh ating? sata Oe 0 aktan™ = spe wahoo eel» ai to et mat — rw 8 4 7. Mies Nad Lala De ae | Ea a, gl 8 ATS etapa NY Mon AV pe ON ee Me ne aga kgs Detach anne Me CANMNS wy apteribcesee Se behbs Rating chase Yn. Detipnystiy 2 Meni wns Derm ateatech « Ne aw eins Siar Myr Sp 
nee nianirmtioe sdthethes inatemelinelbsaietins POA Oe a aig ala abr e * ears te ed =a garde a ou pai mm ee on on mide matin aay ihe Bere cats iee oF Aosta Seer Ab Ne tarnat ter ata Oa AMMAR Abi MMe e, sautten te mete NOG ks sn Sy ae, Se OE a deter tap ile ineredaemtneslnte 
hain Wale, atelier ah Meat Puget naa ora ca FetOyrs Fo" ae OrKahalPan at. © den agauimnke ant aM attr tal pa ieee tA f 2 ester Sm ae ae pate nal eames o° rey Sree Mee ine © wine on meta a, “Ata Mah hapten Morb? Merk Minty, Mettagg.t AE aA SIs S: Mo eNa Mine entenins 3 Sy 237 on Meng ta mane Nir IOCS Ren AOR he hatte ees 
“ mnen: rd Aare Metab aE Bg ne A wet art  P ge eo wale aca — aad Fete sty Bee et Nat fe Me ” rae nS aa Sone, fe ec aw avoantte CD Gane? a Cnhlemaprintlinn ain tr ONapan goa arnt tag hae tes OB Ms Gel ea tae ten Ryan Mead tog «kong mae asthe Dera Aen APNE 00 Poa bee RAR 
ee ee rime te ty padtan OS ta moe aw - = tee ae Ms ts = ie oc Asuna ey eee ere any Lae BF TA Rt Serene Sede Be UL corte aan ne Oy BAe i Rep Atty Ne Carty fea tae com naeaneevn “Onn, Pe ee Men te LINE, Snel Ag 
AE OP on PPA AB NS on ea an Ped adopt a w ee oe » aGh aw Pah “a == es a Lemma : - oat id a oF Mensaabyg frags FaPune ete oy Re ere A stofiy thes oe Rete cmt ete Wet et Se tet ech NKE Leet Py eat Ata ay AINE NCA ON a ig, Contac ed ante Mae 
= eee OM ST ast abana ee Bee FAO He OOK io pail - Sonesta eee fad = dl of Pr thet oa alc — <. ae OES PAsive Rk Leute atyrtadhae "Vd rely eee = Peet MA Pomme eee On NY SMe bee DAte ae rhatote SS Romer 2 emery he shots Ah maine oe = Bo eee orton ~ Pp Nae any 
Maty at ae eee Te heeled we naptahoge 4 - a one Fp ee tP saaP tee = “¢ = Sate i ofr = ove aed ms an oe we ariar ote ry van ae Metre — Ga thee! ae Ratt teen AYR Deke rll Tan teyegene a Bear ewtee tna me tena hes tasite Rema ha eam ane Te tardineap tt tipiomteyriaign 
- Phe ta ae¥ tem ge vet OM shag Sete ona oa eee arte “A « oP, wo) wi Fn ah . Mayan 1% bond e p i gpa ~ - rete we Wh warhy Pret Bie eo cee re antgs eo Oren Beth thatattn ts ene gpl at Bet AOR eile Neen walernw tuple Seaton ands, To th deena <n Smee ey 
mei at Tieaew am oF tat wr ta” Mats alee aot Ban ~ ae oo teal ill nae 's oese ato oe oe. oem Ad | Pn et ots ee ee ee Oe NY Mara dg ha REIN atte WOLD Es hI Nee SLE nce Naeenty, aloe Mgr gates! da Aine Os ie Mi etn tay et 8 Rn Pae SGP RED Me 
we of wtate Ketel we tt ry te ahs te ow = te at Opher (eee i ew = = sad om PO ee eae <4 ce ona: . ee ee ee Oe Rha cies Wy fie aot Ss Take & Wer tetnte ge ete Mew Laan fy Play tle ae vas ae ee Sty ante y 
~ ey al on, oo wiattel ings "oer Maliedien wh ie in ~~ ” Ps =e an” ™ yp ~ —~ 2 Pe = wt te ee ~ ie net / oa no = imemig ete be Ree TAD Be Ne gg eater’ im amma Re Pageant POEL POON iy Meet aims 
wr a ty Pain wt me rete ope 2 a ot tad Pe >. mesa percncae_. soe” we woe aandatall we — seers aay Pigeon ~ an AL we A cl Y ed tobe Preece, Mr MaMa: 8.29 Wiens teal AM atts, eat ne eee ew ee melee sh enn alin eta Yan te tor © ae tale ee Wun Sy 9 
in rae et we Diet” AF OA en ental Ptr ” mad ot ¥ cao : “noth aeibes a het aM a — on cacaits J = ra “ ‘ae a An — andmtta ot UG rwivaes wie” euatinfte « Ge teee nee * > ae mt wet ot Rep wots Ti Tap Sei Capel Ny SLO an PS LL Al Si 
Farragher pe @ | a Vat (hm abet eete Aang at ater one ea oh ae et eae Ore emer aia * a ~~ F : ae -. . rs ile apties Rs geet Abtnn ~~ Aig a th taal. Vs he wep Matyi! AF Male telat, SAN iat eA ee Satpne Mabe me Malan fates Wem Apt ahd Reg Mee eee Mate 
a I att al in Dated ar tn ial eat lad tliat ee en. ee ~ hs ae 7, sai - 7% ams 4 kw F Frye = a. | ne 4 was nes Oi 4a  velert S why Sm Sethetote hae att RAR eye etre ee he Lae ae ete ela sngern te'tape Geet at Os, ATER GI Tn 
" oF ila Poel Sima olla? aeeg ann Pepe ts « ™ at ot ar a Shale kd = Sef natal mae! “: et 2 = “hea aw Ps ee 5 eal Eee > he ot ee we enthorietn Da Sites ARON. aytepAiaha ty tes taltiahitve tan Ranh Are ine st Seakia My ee ee hd 
tm Sa m= - tm a Ming? oat Pate Knead “mit = w A ott & Sd at cal ‘ “ ~ r : - een = wih arertehgye & a ty tw Sar” Sh eatinmn Skate Sarhatne Maen te Mbt TH Lette Deke a regret th AOS igh Hyer Tach Me Tae, = Molnar ey ey <= Sn 
f owrn os, watt faite we Sg. = ul att Eee ne Ne cae -_ «6 arenas aperee 7 ne a ne NAR aN pe iuehsde.  Peftanns Satelite wt be ettneaec te” ah tive Apia toes attay HNaesngn he 
= = ~ wre ss-. te yo wae e ae ed + s « aoe - ee . em a - =%, nd ~ at tarteeh ewe Me & - te nes = Pe Oh ty ble cline Re -enmagrtgt Onew Wp An Ose pele eae = Aalto h-grdetiptre tes 
= ae wt So oh ed a a a a yw eee aeaigeh- ae oe ae a ¢ e OFv tm ae pe ~~ oS ae a en 2 ae a ~ . & . PC Reig Me Tn BG Rea tee = pia taint aay EE Se WA yates 
-” ef ate sede lan w PE AM, te a Se Pr pp bed a é = sue veal wPlerat Ente : ie al ar - ey eat ~ i atin ~ ye Pe on a ed ee a ee enn yt Sateen Le ata Oe al athe tars te 
aN et SPetat el 8 we ie aatiphg= t a °. ans at oe - or * - aor = * ' care =¢ iy ~ ae ee June. Te wow if OVA ee ew eet Rett, ee Mig Mees Bah ele Me Fe AS Sieime Nap ating 
le Fate ether te o 3 we ena dP aa Come bd had * a = ee ey, ee Neventtes tere Sate erie Ne Say tet A Caen artnp ne te airy tan eS Pall Me Nel Ae tetorne ye 
—— Prana ew we onlin aoe ae ~~ eae he othed yw * Py aa - vor wher - ont: * am «< - ee aertettn & MA oye afin gta teal = Heel rt a epvietnaligtteMatMignt «ye SEs sae 
oases tare mek as ar . Oule ° on cues - Swe, w tote i Sy he Oe arr a rr en le ee ee ee Nae eee etn iirda sunkearey read Bitch ane 
ad ~ — em “ Phe re ae ee PA, s ate a - vy aki me -_ m . NG tate ° . & Pot See “ = ‘we Ree Srmeeisn ae! BAe Nye Pe Mee RR een hose abel tamed ter ee ar bn 
ae age Bae meen rt he Se . ° ae F _ _ 4 ae sia nad = Pasee ow aon af meet eet See fete Se? a te Pigg Pn in te ing are ap ee Ey ey mo Yas? rear eae ses 
et em et ~~ Rey wn o tab 5 eave Foal = st) v . - sd ad a - = a ule ON, ange 7 Senate: want 3 e< aw = Ree “qe: Page OY gn Sel aw ~ 
2 men wmatrntede MoS Ket otal wt aref at Rates “ aah bog Ld o we = —“n < aba aie pay Te er eS oe Moe one re Ce ts Renters its tee Me Let a ees crest reat Me rete tent Breen AEN 
peti died eM pale 9 e © 4h - #4 a be nod ~ ~ x te ORC, eae eaeaeal ~ - —— : os eee wert tage Ye - a Paya | Pad Spadina, Se? ree ape PR : > 
ee Pam he Ae tn aon ais S “3 we ? ~ 7 ° - « « a ee weal Renhy een Ne Reem eae met Be Parca. - i A Min gy ey A an Mes, ae Me 
a -*e ” wy at ey ee ol ti ae > ~~“ + 4 af : . —— i, : - se 8 ey - « a ~ He rr 8 tricia te yy et te Satine oe 2 
aoe ge ~ AO, hare = ere aie me _ “at * Ce ~ ie md mao = b= ow we meh he a tt Reyes Sete ma Re Marten A Ti ellag Va cote Ooty ap tr pG eee IAAL CS YONA 
aim Re ae oF tea Fw = = at ner Fw , Mer a, ae Sod lias . in pirate i = ree “ ~ pote S26 eet A - wg Rete ee Bike ln ig et te - 0%. —_ ee tee ot 
Mg = “nae faa ent ane oP I y-ray aed tg ce spe: ai — - > am, niet rn nee at ues Samet — eee Reet s methane: oa a a Ptecgtnee tata wegen. Be <TR Nas — wm 
= ie a -~ "- Pee = eal ch ar Ce ad a = on on, Pei o> & wwe oh : = ame, wer Tengen gen net Male Meena Ny. we Safes 
—— ee al a ~ “ aon a ee if = = Sa ee aw tone ~~ & eney Teo a = et en al ee a Se Se arm nms 
ne ae oe a oly - ee wa Pag i We Ww rd tar PO oe oo es ee —-_ ~ Ned ohn 7 ~ A witgia” 9 wate ernie eevee * te Dor? = here —~ en 
si hed “~ ar rat z= — = z es — emt fee Serene ert wen tet etn ety av be Rote te <a 
os 7 ere wt we - eo, ea r) « = Fa , ~ 2 A. = ie a sf - A emailing aan Paar Wat Se Sg * Se "ace havea Tas 
> —= pe toe atl ator a Ph me - o ~ ene Pee ene met 
on wet or - ° eli pene = =e oot af van tn acts. = < “ oa — Pa be’ ~ * ea am aoe —. Sue oe ee ee — x ~ 
ote oh a pre a = y Paar? af ~ Cl oy ~ - a > A ng fey mR = Meash, eats whe We oes w ween Me Men My ote 
= et ae ae v- “ ~ pela tas — == ed ieee er tasest oe A ~ Bay ~ nemo Mer a“ Femme me 
ont ets - a - Pare ad acto Pes ~ = = =e - a a = Pity, ees 
- ows Nahe Ad a ae a rue cs — = pe y a ~ Ne = “ Re wie ~ wa Pants 
og ae ee cola ™ ae — — at — ae rn % oo Mate aia 
2 are Fm — os . este # a = Pek te Somers & fee Team » nde 
Nore i oe. ae 7s as * = = — | = - Pe > _— 7 aoe a —ts 
- ” ~~ z oa o - a rt a a oe we ee. a Rirat a AAed eo 7 2 “gee es. me ote whe 
~ me te ot oh pas ae Py é an ; a ory . het heat tae ee ee oe = o - r 
i = “ “a Sa ° te ety a. = ~ ae 
- = - ~ww o oa = ae bn wa ~ * tne —o = ee = we 
oe wt A ae Log > _ = re ‘uae - Teo - 
- ote - ~« ee Pj Ps © = <tr eheangngee se pond em * 
Mant an oe. . = =i re Lee “ ~~ Rs - ~~ we weet 
yt a wt ¢ aw pee ee ee wee ” an ‘a ~~ 
Pe we « Pee ow i zs = - = =! g nan e ~ = avrer nate ewe 
oe ate eee ~~? - ae S bad Po re _? .~ pate? As OH Lad 
-_ ae ig - = a = a — < om i aa ey fe oa - s - 
fat a ‘a oe a a ° + ro = wh ~ = a o~ Pangr 
- - es ae = tae, 7 a o “ -— 
: = . ole om = — te = me ate nate “ 7 
ro = — = ‘I we ~ a=} 
* nic =. = oe ~ ngs 
- -* — be C= = - 
» ~ a 
Ser sal lad . 
« 





EO RMOX LIBRARY 
HHA 93943 











NAVAL POSTGRADUATE SCHOOL 


Monterey, California 





THESIS 


A FRAMEWORK FOR SOFTWARE DEVELOPMENT 


by 


oie IGS Vahbeedgulrenere 


September 1984 





Thesis Advisor: Dean Guyer 


Approved for public release; distribution unlimited 


T222822 








ns 


SECURITY CLASSIFICATION DF THIS PAGE (When Data Entered) 


REPORT DOCUMENTATION PAGE 


1. REPORT NUMBER 2. GOVT ACCESSION NO, 3. RECIPIENT'S CATALOG NUMBER 













5. TYPE OF REPORT & PERIOO COVERED 
Master's Degree 


September, 1984 


6. PERFORMING ORG. REPORT NUMBER 


8. CONTRACT OR GRANT NUMBER(a8) 





4. TITLE (and Subtitle) 









A Framework for Software Development 






7. AUTHOR(s) 







Mere C. Hughlett 






10. PROGRAM ELEMENT, PROJECT, TASK 
AREA & WORK UNIT NUMBERS 





9. PERFORMING ORGANIZATION NAME ANDO AOORESS 


Naval Postgraduate School 
Monterey, California 93943 










12. REPORT DATE 5 


September 1984 


13. NUMBER OF PAGES 
101 


14. MONITORING AGENCY NAME 4& ADORESS(/f different from Controlling Office) 15. SECURITY CLASS. (of thia report) | 
t 


11. CONTROLLING OFFICE NAME ANO AOORESS 









Naval Postgraduate School 
Monterey, California 93943 





Unclassified 


15a. OECLASSIFICATION/ ODOWNGRADING 
SCHEOULE 


16. OISTRIGUTION STATEMENT (of this Report) 


Approved for public release; distribution unlimited 


17. DISTRIGUTION STATEMENT (ol the abstract entered in Block 20, if different from Report) 


16. SUPPLEMENTARY NOTES 


19. KEY WORDS (Continue on reverse side if neceasary and identify by block number) 


metrics ,standardization, Ada, maintenance, quality, assurance, stars 


20. ABSTRACT (Continue on reverse side if necessary and identify by block number) 


All sectors of society are confronted with what has been termed the "software 
crisis". As the world's largest single buyer of software, the Department of 
Defense has undertaken major software initiatives to ameliorate software- 
related problems associated with major computer systems acquisition. This 
thesis provides an overview of common problems in both embedded and administra- | 
tive software development and acquisition. It defines quality software in 
terms of its characteristics, and provides the project manager/acquisition 

| Continued 
DD oan 53 1473 EDITION OF 1 NOV 6515 OBSOLETE 


SN - LF- ‘ a ee 
caw En Olsscous 1 SECURITY CLASSIFICATION OF THIS PAGE (When Data Entered) 


a A A 
SECURITY CLASSIFICATION OF THIS PAGE (When Data Entered) 


ABSTRACT (Continued) | 


agency with a set of accepted controls to assure that quality is built in to 
software for improved maintainability. The difficulties and limitations of 
providing accurate estimates in software development are discussed in terms 
of software metrics. A number of DoD current and future standardization 


efforts, including the Army's development of a Military Computer Family (MCF), 
Ada, and the STARS initiative. 


a LS a as ee | ALT 


S/N 0102+ LF- 014-6601 


(RR ta A SS AS 
2 SECURITY CLASSIFICATION OF THIS PAGE(When Data Entered) 


Approved for public release; distribution unlimited. 


A Framework 


Or 
Software Development 
by 


PELewe. sng nilett 
Lieutenant Commander, United States Navy 
Pwoe se 4a) Appalachian State University ome = 


SuUetiecea in Dartial fulfillment of the 
requirements for the degree of 


MASTER OF SCIENCE IN INFORMATION SYSTEMS 
from the 


NAVAL POSTGRADUATE SCHOOL 
September, 1984 


DUDLEY KNOX LIBRARY 
NAVAL POSTGRADUATE SCHOOL 
MONTEREY, CALIFORNIA 93943 


ABSOTRACE 


All sectors of society are confronted with what has been 
termed the "software crisis." As the world's largest single 
buyer cf software, the Department of Defense has undertaken 
major software initiatives to ameliorate software-related 
problems associated with major computer systems acquisition. 
This thesis provides an overview of commom problems in both 
embedded and administrative software development and acqui- 
Sition. It defines quality software in terms of its charac- 
teristics, and provides the project manager/acaduisition 
agency with a set of accepted controls to assure that 
quality 1s built in to software for improved marntaine 
ability. The difficuities and limitations of providing eceens 
rate estimates in scftware development are discussed in 
terms of software metrics. A number of DoD current and 
future standardization efforts are discussed, including the 
Army's development of a Military Computer Family (MCF), Ada, 


and “the STARS 2nd taactid ver 


ide 


id . 


DUDLEY KNOX LIBRARY 


NAVAL POSTGRADUATE SCHOOL 
MONTEREY, CALIFORNIA 93943 


TABLE OF CONTENTS 


CNDROUUCIMIGhMeemeeGmCmic—s © « s <« « © © ¢ s « © @ @ 


A. 
Be 
Cz 


TEE 
Dee 


PAG OGROUNGD apes Namae  « « cs © © « © © «© © 6 ¢ « 
De CC seo Or TW An Geet N DOD 9. «5s « «6 6 = 
DUR PO Serve maP PROACH @. Ss ss) 2s « © 6 es le 


SOD ieee eo S ets © 65 « 6 6 « © « «© «© = « 
EROb eo iomenmoOrT WARS ACOUIS@IION - «. « « « « « 


eA OMEN ODOT Gaco— ss 05 fe sls le 6 « © »© « « 
2e The Multi-source Unified Data 

Dio tEabution me HUDD) Repore. . 2s. . « « 
Salou encavoneovyStem SOitware Study... « 


ASU ners mOnme Cll HOieme ss « © 6 © «© « e« = « * ®» « 


Ae 
ES 


PGi Gi Uso ots sl Nistia 6 « « oe lela « eis «© «© « 
CAUSGo DOr  FOOnmoOrIWARE  SSTiMATING . « » -« 
ee loCmemor enor miat ind EXPCmuSe. i. . . « 
eases ese dt ind as 2 « » © « s « « « 
3. Poor Understanding of What Estimate 
ican Gme—e sts) 6 6 os « 4¢: 6 «s «)'s 6 « « 
4. Estimates as Basis for Incentives ... 
SO ours et > «- elMecmdies ‘ee « % -s 6 0 
1. Halstead's Software Science .....«. -« 
22 NcCabes S@enplexity Measure 37s. . 9s . 
SO may ACOs TENG irs. “s—5 cllte sc leeeamGmnn, «© 6: 6 
Vo BUCCI, <5 0 a ee 
PRE eCOMMCoWE TON, «GM? sMNENs « < -s «oe 6% 
Parone mene atOGCLS  .« <6 <« s « « « 
Sie ino Y ell os « oe Gel 6 «© s © « «© © © % 


10 
10 
11 
14 


16 
16 
16 


29 
24 


26 
26 
a 
Pe | 
a0 


Ze 
28 
Zo 
ag 
31 
SZ 
a2 
Se 
Ss 
35 


IV. 


(AE 


On Ue LY SOFTWARE td] e e e bad * * s e Ld e s s 


A. 
B. 
Cs 
D. 
E. 


Fe 


BACKGROUND 2. 2. 5 « 5 «© «© = oes 


DEFINING SCOPITWARE QUALIT Yee. eee 


CHARACTERISTIC OF SOFTWARE QGUALT aa 
QUALITY ASSURANCE 2 5 2. eee 
IMPLEMENTATION OF A SOFTWARE SOU ae 

ASSURANCE FROGRAM 2 30S) Ss) yo eee 
1.  Procuring Agency Evaluation ... 
2. Design Inspection  s 3.) 
3. Code Inspection <2 eee. eee 
4. Test 2 & 6 Sites oh cu) eis tere 
5. Library Controls 2 een 
PARTING COMNNEN@S > 2. 592 5 eee ee 


SOFTWARE MAL NTENANCE e e e 2 * e € * * e s 


As 
Be 
Gs 
De 
E. 


DCD 
A. 
B. 


CATEGORIZATION OF MAINTENANCE ACI ia 


TANGIBLE MAINTENANCE COST 2 3. Sates 
VARIABLES AFFECTING MAINTENANCE COSTS 
INTANGIBLE MAINTENANCE COStS =. 2 
BUILDING MAINTAINASLE SOPIRAR Eee eee 
1. Structured Methodology = <5 aa 


2. ~ Structured Analysis © ./2 Seco 
Se Structured Design <. . Sue 
4. Structured Programming . ..#..e. 
5. Program Design Language cee 
PARTING COMMENTS © < «© «© Jeuee set eee 


SLANDARDIZATION AND@SPECIF 1G. ONS a-ae 


INTRODUCTION . . 5 «© 6 QRRMRen concur 
SPECIFIC INITIATIVES «. «. 6 sage 
1. Military Computer Family eee 
Ze Ada « «6 ws 0° a) eu euuren ou ntee c cre 


3. Joint Logistics Commanders Workshop 


Be 
i / 
383 
32 
41 


44 
45 
46 
48 
49 
50 
50 


oa 
5] 
@) 
aS) 
oe 
5i6 
a)! 
28 
Shs) 
60 
60 
61 


62 
62 
62 
63 
64 
66 


Vil. SEM cmeeette eres ite @ietits lc 6 « © «© «© @ -s « «© «© »® 
Nee rOVERVY LEW OF STARS ee ne ee ee ee ee ee ee ee 
DeODU TC lL VES ae —CCMECMENC . «© S «6 © « « « 
CeeCRG ANT ZA PUG temsl as) <6 «© « « «© » « « « & « 
Der Cie tin vseereo hire Oo . ws 6 « «+ «6 ~« @ «+ 
Ee eeROGR@ UMP eNAGBIMENT 2. 5 « « « «© «© « © © «© « 
Pom Pee oROVPNGerPERoOONNEL RESOURCES -. « « « «© « 

1. Key Population ASSessment ..... . 
2. Career Structures and Incentives ... 
BRE MeMamGemerrOgrals  ~ sss. s « « « » 
TeOmtommraulecdt tonal SUDtTaSKS . » «6 <« « < 
Cee nO PiiGweenOe hoo LNG TECHNOLOGY as. . . « 
ion onstiwoeuon OF PROCESSING DECHNOLOGLES 
len En peove Business Practices =. 3 =. « » « 
PE EOVerLoOrmusaol Li ty Ss ce. 5 5 
ee ee) > LO) entice is sls so yells a) ss)Nicle) 2 « « 


fees 6 CONCLUSIONS AND RECOMMENDATIONS 2... « « « 
APPENDIX A: GLOSSARY OF SOFTWARE QUALITY ATTRIBUTES 


Mee enDixX Bs HALSTEAL AND MCCABE'S SOFTWARE METRICS . 
Renee oOo LEAD seoOr TWRRE SCIENCE 3 2 « « & « « 
Pemele@capto Ss eCOMPLEXTTY MEASURE « =». « « « « « 


Meee ixX €s STRUCTURED METHODOLOGIES .«. 2. « « « « « « 
Peo TURE DPANALYSIS” 3s « © «© « s+ s s © « © 
Ue LU Smo rotGN! 6 6 «6 © © = © s « « « 
Cee UGC hUR EO ee ROGRAMMING 2 so. « « «© © 6 « 
Dee COG ame PoetGNe LANGUAGE “925 5). « 6 « = & 


PCr Lh ERENCES oes ss «+ «© 8 6 6 © 6 «© @ @ @ 2 


Beemer Oto RU BUTLON List s 2. 2« «© es se we tl lth tl tl hl ll 


69 
69 
71 
73 
Ue 
74 
76 
76 
re 
Ui 
Uy 
78 
a9 
ims 
80 
81 


a2 


S 


87 
87 
92 


94 
94 
ye, 
20 
Ske 


a 


101 


Cost Trends: 


Language Level Values 
Cemparison of Cost Estimating Methods 
Evaluation Factors in Bidder Responses 


Ada Specificationse. 


LIST OF “FABLES 


A Sorting Subrcucines. 


Operator Count 


Operand Count 


Hardware versus Software 


14 
5) 
34 
46 
65 
88 
88 
89 


Ze | 
4.1 
4.2 
4.3 
Be | 
ec 
Ee | 


LIST Or FiGurkeEsS 


Value of Delivered Software . 
Gharae cebboelcoeipee 2 2s « « 
Gori MpaGtaor Changes ....s ss 
PasleGmecGemotructiiecs | <9. su. 
Maintenance Cost as Percentage 
Life Cycle Maintenance Costs . 


Control Flow Graph Complexity 


of Budget 


Pa fi 
40 
43 
48 
513 
54 
Sz 


I. INTRODUCTION 


A. BACKGROUND 


"and a good south wind sprung up behind; 

The Albatross did Petlone 

And te day, f£0@ =Locdmen play, 

Came tc the Mariner's hollo!? 

God save thee, ancient Mariner! 

From the fiends that plague thee thus!-- 

Why lock’st thou sSo?=="atieny cross-vo vee 
I Shot the Albatross." [The Ancient Mariner ot. 


The albatross around today's program manager neck is often 
the sortware subcomponent of major system acquisitions. Cost 
overruns, schedule slippages, and loss of program control 
have keen the penance for those project managers who have 
failed toe provide for software with the same intensive and 
continuing management typically rendered its hardware 
counterpart. 

Software 1s an intangible product that defies descrip- 
tion in an engineering sense. Only a few software products 
have ever started off with clear, unambiguous, and defini- 
tive requirement specification. Schedules and costs are 
often dictated by the system acquisition milestones and 
reviews, and not necessarily associated with the phased 
software development methodologies advocated by what has 
keen termed "software engineering!". Many of the specific 
problems that surround software development and acquisition 


will be discussed in detail in the next chapter. 


1The kev I ASN hE of software SSSI | are  ( 1) 
well-defined methodology that addresseS all ases of the 
software life cycle, aa an established se of software 
components to document and show traceability from one devel- 
opment step to the next, and (3) a set of predictable mile- 
stones that can be reviewed as needed [Ref. 1: p 15]. 
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In the majority cf guidance and managerial principles 
available to assist the program manager are directed at the 
hardware end. Software is the "new kid on the block." It is 
that part of the system that 1s seldom understood and often 
mismanaged during system acquisitions. Computer hardware, on 
the other hand, has undergone remarkable improvements in 
function, size, performance, and relative cost. Several 
hardware generations have emerged in the course of a single 
human generation. Yet, software has experienced more notice- 
able growing pain. The gap between hardware-- and software 


technology widens. 


B. THE COST OF SOFTWARE IN DOD 


There are two general classifications of software within 
DoD. The first of these is that of the more-traditional, 
administrative type cf software used in business applica- 
tions. This type of software iS typically supported by 
commercially available computer that can Support a variety 
See applications, i1.¢é4., Automatic Data Processing (ADP) 
systems. The second classification is embedded software. 
Embedded software is normally designed to operate as an 
integral part of nen-ADP systems, such as DoD tactical 
systems. The most significant difference between these two 
Classifications of software rest not in the development and 
Maintenance practices, but rather in the frame work in which 
they are each procured. 

The procurement authority for Automatic Data Processing 
Equipment (ADPE) and its supporting software and services is 
vested in the General Services Administration (GSA), as 
directed by Public Law 89-306, 40 USC 759, the "Brooks 
eee. Within DoD, ADPE is under the purview of the 
Assistant Secretary of Defense (Comptroller). Weapon system 


software is under the cognizance of the Office of the 


ue 


Undersecretary of JLefense (Research and EnRgineeripeg ee 
Although there is a distinct dichotomy of cognizant organi- 
Zational structures regulating the acquisition of ADP and 
NOn-ALP SoLtwane, the managerial and software engineering 
principles which govern each step of the software life cycle 
are, in fact, quite Similar. Therefore, the common set of 
tools, methods, and methodologies advocated in this thesis 
apply to both ADP and non-ADP software. 

In writing this thesis, it was noted that the majority 
of available DoD guidance for the control and acquisition of 
software projects was in support of tactical systems, with 
the vast majority being authored for the United States Air 
Poree. This 1s not suprising since it has been estimated 
[Ref. 2: pw 7] that of the $12 billion that DoD waite eee 
for software in 1985, over $10 billion will be for embedded 
software, with the U. S. Air Force accounting for approxi- 
mately half of the expenditures. 

Not only does emredded software represent the largest 
component of total software costs in DoD, it is also plagued 
to a proportional degree with many of of the software- 


related problems, which M.M. Lehman so aptly describes as 


"a Motiey. cCollectien. of ee OS isolated methodolo- 
les and techniques associated through an experience- 
ased, . but otherwise a Di tire aay sequence Ou 

much-discussed process phases". [Ref. 3 3: ps 3] 


At this point, it is important to recognize some of um 
some of the program characteristics that add to the complex- 
ities of DoD's embedded software. These include: [Ref. &: 
pe 77} 


2Due in large part to the provisions set forth in the 
Subseguent Warner Admenment, the policies and procedures set 
Louth In the  rooks aie does not extend to the tactical 
software used in DoD weapons systems. 
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=——errOgrdmecize-—-Otten in excess of a million lines of 
code. 


-- real-time, operatin requirements requiring response 


time in milliseccnds. 

~ BEQSES"S BLGEs on” Gvee aa expected essal-Live or tes 
in excess of twenty years. 

-- guaranteed reliability due to the tight (and many 
cibvits. user fe the population’ that the ‘eystle ts 
designed to protect. 

Sond Ts'o hs I we part of the universe which they model 

As ccmpared to other software applications, such as ADP 
Sesadministrative computing, DOD nission-ecritical software 
is more complex, less understood, more unstable, and must 
operate in extreme environmental conditions. Yet it is 
essential that DoD software be reliable, adaptable, and 
affordable. To achieve these objectives, many problems, of 
roth a technical and managerial nature, must be overcome. 
Symptems of these problems include slippages in wearon 
delivery schedules, system failures, overbudget programs, 
and inflexible systems will be discussed at further lengths 
ame Chapter II. 

DoD 1S recognized as the world's largest buyer of soft- 
ware. Based on various estimates in recent literature, it is 
calculated that DoD will spend approximately $12 billion for 
Eomtware 1n i985 [Ref. 2: p. 5]. Table 1 iilustrates the 
percentage of total computing system costs of hardware as 
compared to software for all of DoD computing systems. 
Software cost reflect all aspects of the software life 
cycle, including: design, development, testing, operations, 
and maintenance. The ratio of hardware to software has 
reversed itself frem 4:1 to that what is expected to 


eeeeoach 1:9 next year. [Ref. 2 ; pp. 5 - 6]. 


The high cost of acguiring software has naturally 


caused ccncern in beth DoD and the Congress. Literature 
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TABLE 1 


Cost Trends: Hardware versus Software 


(percentage of total cost) 


a) 


85 
1955 1970 1979 (Estimate) 
Hardware 83 4S 25 10 
Software 17 aS qs 90 
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abounds with studies and recommendations related to software 
development in DoD. MThere 1s not a shortage of sage advise. 
The need for improved managerial controls and software 


development practices has been recognized. 


C. PURPOSE AND APPROACH 


A major goal of this thesis is to present a consolidated 
review of major DoD efforts aimed at reducing software- 
related problems. Both management and technical issues will 
be addressed. This thesis makes no pretense that it provides 
the program manager with all of the technical background and 
controls needed to assure the timely delivery of quality 
software within budeget. Rather, it focuses on key and 
"high payoff" issues involved in managing the acguisition 
and development of software. This thesis also addresses 
several DoD initiatives which promise to significantly alter 
the framework in which software is developed and maintained. 

Chapter II identifies many common problems associated 
with contracting for general computer software by Federal 
agencies. It also identifies major contributing factonpomae 
DoD weapon system software problems. A common denominator in 
the formulation of the many software problems is the lack of 


estimating expertise by which program measSurementS can be 


14 


defined. Chapter III discusses software metrics for defining 
guantifiable measurements. 

The delivery of good software is an implicit, but most 
elusive, goal in software acquisition. Chapter IV defines 
"good software" through a set of guality software character- 
istics. It also provides a series of controls to be utilized 
in the implementation of a quality assurance program. Major 
dividends from quality software are realized during the 
post-development phase of software, the maintenance phase. 
Chapter V analyzes the tangible and intangible costs of 
software maintenance, and addresses a number of software 
engineering principles through which the costs of mainte- 
hance can be greatly reduced. 

Chapter VI and VII review a number of software and 
computer- technology standardization initiatives to under- 
taken within DoD. Perhaps the most significant of these 
initiatives is the STARS (Software Technology for Adaptable, 
Reliakle Systems) progran. 

Finally, Chapter VII provides this thesis' conclusions 


and reccmmendations. 
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tie THE SOFTWARE CRESS 


"The problem of the 1970's was to reduce the cost of the 
eee re LURE tren. needed to store and process 
a GQeeee 

"The problem of the 1980's is different. Now we must 
reduce the ccst cf electronic  solutrvens. that ase 
reducing the cost you incur. in uSing our _ devices to 
buiaid a product. Solvingmenss roblem will require a 
shift from the component integration of the  1970"siue 
concentration of system level integration in the 1980's. 
"We can now that about putting power of a mainframe CPU 
on a single chip. This buys you nothing as a Customom 
however, unless. you can use that power. Hardware is 
Sd See potential; it must be harnessed and driven by 
software to be usetul.” [Retsil 


The preceding statement was made by the president of one 
of the largest manufacturers of computer hardware. iG 
succinctly summarizes the shift in technological emphasis 


from hardware to software. 


A. PROBLEMS IN SOFTWARE ACQUISITION 


To the casual observer, the successful management of a 
software development project may seem a simple process. All 
that is needed are {1) well-defined reguirements, (2) real- 
istic cost and schedule estimates, and (3) the right gquan- 
tity of personnel and hardware at the right ))tmmee In 
actuality, each of these elements seldom, if ever, happens 
by themselves, much less together. 

The management of software development projects have 
historically been plagued by a myriad of problems, both in 


the private and government sectors. 


po ee 


—_— = 


Report 


Recently GAO reported to Congress [Ref. 5 : pp. 1- 


84] a number of probiems that Federal agencies have 
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encountered in contracting for computer software as an 
meretmnative to in-house development. Means for improving 


these deficiencies were also recommended. 
a. Scope of the GAO Report 


GAO sent guestionnaires to 163 software 
Semrracting firms and 113 Federal project offices that had 
experience with software development projects. The purpose 
of the guestionnaires was to attempt to identify common 
problems in software development contractual process. and 
what, through hindsight, might have been done to prevent or 
improve development efforts. 

GAO examined nine cases of software development 
in detail, some of which had attracted GAO attention because 
they were known failures. Only one of these nine cases 
yielded a software product that could be used as delivered. 

The actual combined total development cost and 
time for the nine cases almost doubled the estimates of 33.7 


Million and 10.8 years. 
ke. Common Causes of Software Contracting 


The nine cases that were studied in detail 
illustrated many of the same causes of difficulties that 
respondents to the GAC's questionnaires had identified. The 
most significant of these findings will now be described: 

-- Federal agencies contract for software development 
with little specific guidance. 

Guidelines for software development promulgated 
by central agencies are primarily aimed at the technical 
aspects of software development. Very little guidance is 
frovided in support of the contractual process. 

Basic responsibilities of the central agencies 
are set forth in the Brooks Act, Public Law 89-306. The 
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Office of Management and Budget (OMB) is prescribed general 
oversight of Automatic Data Processing (ADP) activities. 
Much of this responsibility has been delegated to the 
General Services Administration (GSA) and to the National 
Bureau of Standards (NBS). GSA is delegated the respfonsi- 
bility for ensuring cost effectiveness in the selection, 
aCGuasit7oOny, and utilization of ADP resources. GSA's 
guidance for the management of ADP resources is contained in 
subpart One of the Federal Property Management 
Regulations.3 Policies addressing the procurement of and 
contracting for commercially available software is provided 
in Federal Procurement Regulation 1-4.11. GAO's review of 
both of these documents revealed that there is very little 
actual guidance directed at the specific contractual manage- 
ment for engaging in custom software development. 

Although NBS is tasked with developing technical 
Standards and guidelines, OMB has indicated that NBS is also 
responsirle for investigating and assisting in software 
system developments. Although NBS representatives advised 
GAO that their resfensibilities involved managerial and 
contractual activities for system development, NBS‘ emphasis 
has been, and will ccntinue to be, on the technical aspects 
of system development, Such as the standardization oe 
government-used Higher Order Languages. 

-- Agencies overestimate the Shade of their own prelimi 
Nary work before they contract. 

GAO found two primary reasons why agencies 
contract out for software development instead of doing it 
in-house. The first is that many of the agencies lack suffi- 


cient quantities of, or properly skilled, personnel tome 


3As of 1 April 1984, DoD regulations. concerning the 

ag Wasi pao nea Oe ADPE resources and services Pr Cv eee 

contained in the Defense Acquisition Regulations (DAR) have 

been replaced ae DoD supplements to the Federal Acquisition 

Rega gee (FAR); specifacally, Subchapter 2, Panu 7a 
e ste 
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the work. Secondly, the software is often needed sooner than 
it can tke produced in-house. Often the initial sters of 
software development, such aS requirement analysis, are 
started in-house prior to contracting out for the continued 
develcpment of required software. Two common problems have 
been observed in this context. First, the agency may overes- 
timate the amount of work already achieved in-house 
secondly, the agency's preliminary work that is turned over 
to the contractor may be inadeguate requiring that it be 
done again by the contractor. 

Overestimating the stage of software develcpment 
before releasing it toa contractor is likely to result in 
additional costs to the extent that any cost benefits that 
might have been gained from the development project are 
forfeited. It is critical that precise methods for measuring 
preliminary in-house work be used in order to achieve real- 
istic cost and time estimates. An accurate identification of 
the stage of system development is vital in order to frop- 
erly determine the tyre of contract to be utilized. If, for 
example, the agency has completed all the preliminary devel- 
opment stages required prior to the commencement of coding, 
mmen a firm—-Lixed price contract for the coding effort might 
be the most suitable. If, on the other hand, a systems 
detailed design has not been completed by the agency prior 
to entering into a contractual agreement, then a phased, 
cost-plus- fixed-fee type contract would likely be more 
Suitakle since the exact scope of future efforts is net yet 
known. 

If agency work that is passed on to the 
contractor is later fotnd to. be inadeguate, or less than 
originally estimated, much of the work may have to be redone 
by the contractor. In doing so, there often is a tendency to 
attempt to save as much of the original work as possible in 


order to remain within the cost and time ceiling mandated by 
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the contract. This is likely to compromise the design of the 
new system, resulting in a less efficient system that 
Mandates higher operating and maintenance cost for the 
remainder of its Late vevele. 
-- Contracts fail to stipulate what constitutes sSaciog 
factory penrormance. 

Failing to stipulate what constitutes Satistage 
tory performance by the contractor makes it ditt iculjeyeee 
not impossible, tO “Glaam DOOD) Peenuaen performance. 
Furthermore, it reduces the probability of a satisfactory 
end-product. Many dispfutes over contractor performance could 
be avoided if adequate system specifications and testing 
criteria are identified in thevcontract. 

Other general requirements and constaints that 
can usually be identified at the start of a software project 
criteria for software expandability, documentation stan- 
dards, maximum computer resources allowable, maintenance, 
and program transfer capabilities. 


-- Agencies eRe overcommit themselves, and fail to 
adhere to strict phasing to control Contracron—. 


Phasing divides the development effort into 
logical and manageable work phases. One of the most effec- 
tive controls availarle to an agency is in the contractual 
identification of phases, coupled with manadatory agency 
review and approval following each phase aS a precondition 
to the ccntractor's ccntinuation of subseguent phases. Other 
advantages associated with phasing include: 

-- Identification of milestones and timetables to 
monitor the progress of the project, allowing for 
the initiation of corrective actions in a timely 
fashion. 

-- Systematic and orderly development of software. 


-~ Control of funds based upon Guality and Saceere 
abizity o£ contractor's wore 


-- Increased assurance that should development efforts 
are being used. 


-- Improved communication between the sate and the 
contractor leading to the increase probabilig 
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PiitmelememlcractoOn rum, Understands the agency's 
requirements. 


-- Ccempleted phases provide an adequate base upon 
which subsequent phases can be built. 


paack sof agency management during Contract 
exeCurcilonl. 


An excessive number of system changes were 
requested by the agencies in the cases studied by GAO. These 
agency-initiated changes ranged in scope from minor require- 
ment adjustments to re-resign of the entire system. Many of 
these changes requested and made during the latter phases of 
development and contributed significantly to cost and 
schedule overruns. 

Project managers should be aware of the need for 
a well-defined problem statement and the undermining effects 
that changes have cn software development. Changes, as 
compared to the original requirement specification, are not 
usually as thoroughly researched and may cause unforeseen 
and rippling effects on other parts of the systen. The 
systematic and logical flow of contract phasing may be lost 
due to the need to modify work that has already been 
completed and approved, obscuring the visibility of the 
project's status. Furthermore, excessive changes make it 
@rericult to hold the contractor accountable for the initial 


terms of the contract. 


-- Agencies to not adequately inspect and test software. 

As depicted in figure 2.1, most of the software 
delivered in the cases studied was of poor quality. Reasons 
for this poor guality was evidenced in all phases of devel- 
opment. Quality assurance must be tied directly to the 
contractual process. Higher guality software can be 
obtained if the contractor maintains quality assurance func- 
tions in a number of software development areas. Specific 
examples of these areas include configuration management, 


testing, program design, documentation, and working tasks. 


aa 


The latter of these area, working tasks, iS a means for 
assuring that procedures are in affect for subdividing san 
total work effort into segments andl asSlgnhing@responsioi ties 


for the initiation and Completion of Bion 


aaa aaa aE Be BES SES aoe a RES SC i a a 


NINE SOFTWARE DEVELOPMENT CONTRACTS TOTALING $ 6.8 MILLION: 
WHERE THE MONEY WENT. 











SOFTWARE 
DELIVERED BUT NEVER 
SUCCESSFULLY USED 
($3.2 MILLION) 








SOFTWARE 
PAID FOR BUT 
NOT DELIVERED 
($1.95 MILLION) 







SOFTWARE USED 
BUT EXTENSIVELY REWORKED 

OR LATER ABANDONED 
($1.3 MILLION) 








SOFTWARE THAT COULD BE SOFTWARE THAT COULD 

USED AFTER CHANGES BE USED AS DELIVERED 

($198 000) ($119 000 out of $6.5 pe 
Figure 2.1 Value of Delivered Software 


The GAO report concluded by stating the need for 
improvements in contracting for custom software development. 
Recommendations were made aimed primarily at GSA and NBS for 
both improvements is both procurement and technical areas. 

GAO further recommended that GSA and NBS work 
together in designing model contracts of various types. 
These contracts would have sample clauses for covering the 
withholding of payments, testing, etc.. Agencies would used 
these samples to extract those clause which best fit there 
particular requirements. 

The last recommendation that GAO made was that 
Federal agencies that extensively contract for software 
development "should train project managers in appropriate 
software, contracting, and management skills." [Ref. 5: p. 
29 | 


2- The Multi-source Unified Data Distribution (MUDD) 


The MUDD Reppert [Ref. 6 3: pp. 1- 28] should be 
considered "required reading" by all present and future 
project managers overseeing software development. It isa 
case study of Navy software development practices. The 
report is based on over 30 interviews with of those respon- 
sible for the development of Navy systems. The year-long 
study uses the development of the fictional MUDD system 
under development to mirror many of the reguirements of Navy 
tactical systems either in operation or under development. 
It chronicles and analyzes the decisions made on the soft- 
ware development effort. The MUDD Report concludes witha 
set of recommendations to Navy program managers for avoiding 
the pitfalls described in the report. 

The issues brought to light in the MUDD report are 
germane to those probtlem areas found in large and complex 


System development efforts which typify many DoD programs. 


PLS 


An adequate Summaticr can not be given of the MUDD report 
which can do it justice. It should be read in its entirety 
for a full appreciaticn of Weiss' recommendations which are 
directed at problem areas that infest the fictional MUDD 
System development. Most of the recommendations center on 
various types of interfaces, such as the interface between 
the Navy and contractor, interfaces between people, inter- 
faces between and within systems, and interfaces within the 


Navy. 


3. DoD Weapo 


System Software Study 


The John Hopkins University Applied Physics 
TapCEaAtonry ss (APL/IAU in Con JuUNnCt Lon ee wise the MITRE 
Corporation, conducted an extensive study of the management 
of weapon systems software under the auspices of the Office 
of the Secretary of Lefense [Ref. 27]. The MITRE and APE 
study team reviewed the findings of ten previous 
DoD-sponsored studies relating to software. The MItka 
Corporation concluded that the "major Contributing =i ae cma 
weapon system computer software problems was a lack of 
discipline and engineering rigor applied consistently to the 
software acquisition activities." [Ref. 7 : p. 50] Otheus 
more specific, findings of included in the MITRE/AFL study 
included the following: [Ref. 7 : ppeagoUe omg 

--Frequent contributors to software cost and) senedume 
growth include: (a) poorly formulated initial software 
reguirements; (bf) Specs SG requirements and reguire- 
ment growth during the development phases; (c) \tadiee 
starts and need to educate involved ofganazatvene 
before useful output can be obtained; (d) 1neflicvene 
use (proliferation) of already existing resources; (e) 
inefficient testing Yau VerificatioOngcaeus and 
methods; and (f). improper use of standards and 
guidance documents in specific procurements. 

--There is a general need for better identification of 
software terms, measures of software qualities, and 
the methods for measuring then. 

--Software technology improvements particularly aimed at 
Gove oP ag a software engine eae discipline are bein 


made by PS Ei academia and the services bu 
LEGuitne Papell ca richmato real military systems (in 
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PGi Totem ura OeCratoOry OF experimental systems) for 
evaluation and confirmation. 


The study resulted in a series of 17 recommenda- 
tions, each of which was directed at a specific preblen 
area. A sample of these recommendations includec: [Ref. 7 : 
poe D0 — 51) 

--Specify that major computer software involved in 
weapon system development be designated "configuration 
cle and be deliverable during full-scale develop- 
ment. 

--Use top-down design, i.e specify the use of _ modular 
software architectufe and an orderly hased design 
aEEtoaes that defines the higher levels of the program 
and then rogresses to design and test successively 
ewer levels. 


--Require the contractor to apply a highly disciplined 
set of engineering practices. 


--Establish_a commcn set of requirements and criteria to 
€ applied...by all services. 


--Prepare a series of handbooks of. guides covering 
important aspects of software acquisition. 

While extensive progress has been made in DoD toward 
addressing many of the problem areas noted in the preceding 
studies, much work remains to be completed. Specific correc- 
tive actions that have been adopted, or which are presently 
in the formulation stages, will be covered in this thesis, 
particularily in the chapters addressing standardization and 


the STARS software initiative. 


Za 


III. MEASURES OF CONTROL 


A. BACKGROUND 


"You can't contr¢cl what you can" meaecmmee: f{Ref. Gime 
P. 3] A disparity exists between the software manager's 
definition of what constitutes a project's success as 
compared to the user's perception of the Same project. With 
software projects resuiting in utter failures or cost over- 
runs of two, to three, times the original estimate, software 
managers often consider their projects a success if overruns 
are kept below 30% and when "most" of the delivered lines of 
code are considered "usable" by the end-user. 

DeMarco [Ref. 8 ; p. 4] writes that many projects fail 
eventhough the project managers have excelled in those 
Characteristics that he associates with good management. 


These characteristics include: 


--project staff members that are highly motivated 
=—clear understanding of thesicsuce 
--adequate grasp of relevant technologies 


~-evident capability in the political sphere 


Demarco attributes the failure of these project managers to 
the fact that they have Simply failed to meet the original 
expectations of the project. He is convinced that in often it 
is not the fault of the project team, but rather the fault of 
"inflated and unreasonable expectations." When expectations 
exceed what can be delivered, the project 1s doomed to 


failure. 


26 


BE. CAUSES FOR POOR SOFTWARE ESTIMATING 


Estimating 1S at the core ope chek: dasteiraculties 
surrounding software projects. Feedback is essential for 
control. Feedback provides a basis for comparing the actual 
project's progress against original expectations. These 
expectations were formulated on estimates. Main causes of 
poor software estimating are as follows: [Ref. 8 ; pp. 9 - 
7 | 


The average software manager will typically rate 
himself/herself well below average aS an estimater. The 
underlying reason fcr this is simply lack of estimating 
practice or experience. 

iienanounewOL ac¢tiial estimating that a typical soft- 
ware Manager is involved in will normally take up less than 
3% of his/her time. Most software projects may call for 
estimates at their beginning, and maybe once-a-month or 


prior to management review thereafter. 


2. Biases in Estimating 


Fersonal biases create a strong tendency to underes- 
timate one's own potentials. However, when objectively esti- 
Mating another's potential, then most of these biases are 
Minimized. DeMarco suggests an obvious approach to avoid 


this phencmenon by stating 


"Wheever does the estimating. for a project must_ be 

Someone whose entire ego involvement iS in the ey 

PR Ec S-ePicwe,eratuuen than an the project itself.... 
ef. 
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3- Foor Understanding of What Estimate Means 


At the very heart of probability theory is the esti- 
mation of "odds" of the occurrence of a certain event. Yet 
software estimates are often void of any explicit probabi- 
listic assessment which may govern them. This observation is 
closely linked to preceding subsection on the personal 
rkiases involved in estimating. Should a software manager be 
asked the probability of finishing a project, say, 20% later 
that s/he originally estimated, an answer (right or wrong) 
will freely be given. On the other hand, should the same 
person be asked the probability of completing the project 
earlier than originally estimated, the estimater will likely 
give it a zero probability. This represents what DeMarco 
[Ref. 8 : p. 14] terms 


Default Definition of "Estimate" 


= ee Se [See eee ee ee = =< — = SS = SS 


An "estimate" is the most optimistic prediction that 
has a noh-zZ¢éro probability of coming = eeu. 


Instead, DeMarco [Ref. 8 : pe 22] proposes that an estimate 
should ke defined as "a prediction that is equally likely to 
be above or below the actual result." This definition, by 
itself, does not sclve the estimating problen. It does, 
however, offer a basis from which to start examining meas- 
ures and other components of estimates which will Le covered 


IN ths chapter. 


4. Estimates as Fasis for ineontuges 


—_—— —— 


Cften estimates are used to establish gcals for 
performance. When used in this manner, the software manager 
is likely to establish estimates on previously established 
goals. To serve as a motivational tools for the development 


staff, the goals are set at unattainable levels. 


Inte 


Many managers view goal-oriented estimates as_ the 
supreme motivational mechanism for their overly-optimistic 
development staff whcse self-esteem is placed on the line in 
the pursuit of unachieveable goals. As the development staff 
is driven toward the completion deadline, the ultimate 
wactim Of Of this motivational strategy is the quality of 


the finished product itself. 


an SOFTWARE METRICS 


The first part of this chapter has done little more than 
point out many of the ill-fated approaches which have tradi- 
tionally been used tc control software. These approaches 
were principally qualitative in nature, having no formal 
mathematical basis. Yet, intuitively, a direct relationship 
exists between software guality and guantitative software 
characteristics such as modularity, Size, and — Logical 
paths. As such, software metrics nave been advocated by many 
authors as a preferred means for deriving inputs for the 
estimating process. 

This section will examine two of the most popular theo- 
ries in software metrics that have grown out of the forna- 
tive years of software engineering: (1) Halstead's Software 
Science Theory, and (2) McCabe's Complexity Measure. 
Appendix 8B provides sample algorithms and respective 


mermulas Lor each of these two theories. 
1. Halstead's Software Science 


The first set of metrics to be reviewed were devel- 
oped by Maurice K. Halstead [Ref. 9]. Instead of using 
"lines of code" (LOC's) to describe the size of a module, 
Halstead breaks each line down into a series, or grceup, of 
symbols. Each of these symbols can be classified as either 


mime OPerator® of as an “operand.” An operand is a single 


29 


symbol or group of symbols that identifies the constants or 
variarles that the mcdule uses to implement its algorithn. 
An operator is a single symbol or group of symbols which 
affects the value of the operand. Operators also impact the 
seguence in which the operation takes place. 

Criticisms ccncerning Halstead's theory of measuring 
through the use of operators and operands were quickie 
registered. The majority of Halstead's work evolved around 
algorithms drawn from Algol and FORTRAN. In other alges 
rithmic languages, the definitions of operands and operators 
are not nearly as clear. Halstead also omitted declarations 
and comments from his calculations--a significant portion of 
the widely-used COBOL language. Other studies, however, have 
shown that the additicnal declarations and comments actually 
brought the estimated program length closer in line with the 
actual, compieted program. In any case, it is important to 
identify the operands and operators of an algobithieae 
language to establish consistency. This function Can Ofgem 


be determined by a compiler, through which the operators and 


operands are explicitly defined in the final machine 
language product. Cuestions abound on the derivation of 
Halstead's formulas. The validity of his experiments has 


been guestioned because of the small sample size, the small 
Size of the programs used, and the subjects used were 
college students vice experienced programmers. 

Halstead profoses that each language can be categor- 
ized by a language level, 1, which will vary among 
languages. The variances are closely linked to the level of 
abstraction by which a procedure can be specified. Halstead 
assigns a constant language level for a particular language, 
which is in contrast by to the recent works that show that 
language level is a resulting product of both the language 
and the programmer. Table 2 provide the language levels 
values that have been empirically derived for five common 


languages [Ref. 1: p. 166]. 
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| Flt (<i‘OSCOC~sdY 
TABLE 2 


Language Level Values 


Language: Mean l 
English pose Ze 16 
BL 1 rps 
ALGCL/66 22 
FORTRAN 1.14 
Assembler 0.88 


| 
to 


Tivol ouarn ele, Gict owl aceeee os, 208) —~320)] McCabe 
proposes a complexity measure of software which is based 
upon the control flow representation of a program. Through 
the analysis of several FORTRAN programs, he illustrates a 
high correlation between the intuitive complexity of a 
program and his proposed graph-theoretic complexity measure. 

McCabe's software complexity measure 1S preported to 
measure and control the number of paths through a program. 
The primarv difficulty is this regard is manifested through 
backward branches which may possibily result in an infinite 
number of paths during program execution. Consequently, 
uSing a path count to measure program complexity is inprat- 
ical. However, the complexity measure can be defined in 
Mmemns OL baSic paths, that when taken in combination, will 
measure every possible path. 

As compared to Halstead's metrics, McCabe's 
complexity measure can be applied during the earlier stages 
of software development since it is not dependent on the 


measurement of code. The cyclomatic complexity measurement 


Sel 


provides an evaluaticn tool by which "goodness" of a module 
can Le reviewed following its detailed design [ Ref. 1: p. 
utes) |e 


De SOPIWARE COS ZENG 


The role of software in the military and private sector 
has grown considerably during the past decade. During the 
infancy cf computing, software cost amounted to only a small 
percentage of the overall computer system. Today, software 
is the frost significant portion of moSt Computer Systeme 
Accurate estimates cf software development cost seldom 
occur, with the final costs normally running considerably 
higher. There are two fundamental problems which make accu- 
rate estimates of software development costs most difficult 
(Ref. 13 : p. 45]. These are: 


-- the high risks and uncertainties involved in software 


development 
-- the lack of a quantitative data base for revious 
Gost aor ss anes ristid le =COS vce ica, lessens 
€arne 


In spite of these significant problems, cost estimate are made 
and will continue to be made with varying degrees of accuracy 

This section will describe three current methods of cost 
estimating and provide a table for their comparison based on 
application (Ret. 13°: Dew loe ee 


1. Analogy 


This method estimates the costs for anew system 
based upon the the costs of a Similar system. The cost esti- 
Mate is adjusted to compensate for any differences between 
the two systems being compared. The analogy method is fairly 
Simple provide accurate cost data for the existing system is 
available and the development methods and resources are 


Similar. 
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2. Decomposition 


As the method name suggests, a system is broken down 
into ccmponents and subcomponents until the level of decon- 
position makes it fEessible to estimate the costs fairly 
accurately. One apprcach of decomposition uses the analogy 
method previously described. In this approach, the process 
of decomposition is effectuated until the resulting level of 
decomposition can be compared with a similar component which 
already exists. A sécond approach of decomposition divides 
the system into comfenents for which a level of effort can 
intuitively be estimated for each kind of activity that is 
needed to produce that component. This latter type of decon- . 
position normally defends heavily upon the technical knowl- 
edge and experience of the estimater The preassumption that 
underlies this method of cost eStimating is that the costs 
for small systems, or components, can be accurately made. 
The total system is perceived as the aggregate total of its 


subsysten. 


3. Farametric Models 


As with the analogy method, the parametric model 
approach to cost estimating is also heavily dependent on the 
accumulation of past and accurate cost data for software 
development. Analyses of cost data permits the identifica- 
tion of cost variables and a guantification of their rela- 
meonshiip to cost. Any new cost estimate can be derived by 
estimating the assigned values of the cost variables, Once 
this is done, the cost can be calculated using the equations 
which express the cost estimating relationships. The advan- 
tages of this method is that it allows for a rapid deterni- 
nation of cost estimates, using parameters whose values can 
be easily modified. 


Sys) 


Table 3 [Ref. 13: p. 16] provides™a conparicoames 
the three cost estimating methods discussed. Combinaticns of 
two, Or more, methods may be used together, or separately to 
test the validity of an estimate. Resultant differences are 


adjusted to arrive at a reasonable estimate 


TREE 3 
Comparison of Cost Estimating Methods 


TYPE DESCRIPTION 


Analog Compare to prior system 


Decomposition Divide into parts, 
aGuivi ties 


a i ot, aS ite fey, ia RO fee EE pts, 


Parametric Eguations based on prior 
Models data about cost relation-— 
Ships 
TYPE GOOD FOR 
Analog « Similar systems with 


Similar resources, 
development process 


Decomposition - resource allocation 

-.« unique systems 
Parametric = tapid esvimarian 
Models - eStimater's inexperience 


in software development 
« eEStimating fisk 


ce ce ee ee eee ce eee ee ce es es es ee ee ces es ee ees ee ees ce ces ee es es es ee ee eee ee eee 
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AD FOR 
Analog - unigue systems 
. different environments 
Decomposition - initial estimates with no 
design ; 
« Lapiad eCStitarroen 
Parametric - systems different fron 
Models data sample 
4 pea COrrelatedsdaca 
ase 
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Automated costing systems provide another option for 
estimating. In these automated systems, the characteristics 
of the development organization, such as the staff's expferi- 
ence level, and characteristics of the software to be devel- 
oped are described. Cost estimates are derived from this 
miput data. As with the other three manual methods, the 


derived cost data will only be as good as the empirical data 


upon which it is founded. If no historical data exists, 
then the validity cf the cost estimates is, indeed, 
guestionable. 


Ee. CHAPTER SUMMARY 


McCabe's and Halstead's software metrics remain a 
controversial topic. But they do represent a revolutionary 
approach toward providing software managers with gquantita- 
tive functions for estimating many heretofore eluSive char- 
acteristics of software. The validity of Halstead's 
experiments have yet to be significantly tested. For those 
tests that have been performed, the size of the programs 
were generally small, and the subjects were college students 
vice professional software developers. 

AS compared to Halstead's metrics, McCabe's complexity 
measure can be applied during the earlier stages of software 


development since it is not dependent on the measurement of 


code. The cyclomatic complexity measurement provides an 
evaluation tool by which "goodness" of a module can be 
reviewed following its detailed design. Despite the criti- 


cisms that normally akound the proposition of new theories, 
roth Halstead's and McCabe's metrics represent a giant leap 
toward adding guantitative measurements to a discipline that 
has defied then. 

The military's justified and growing concern over 


frequent cost overruns for software development is forcing 


a5 


changes in both the management and development of software. 
As such, new reguirerents and changes can be expected that 
will provide a more uniform and better control over cost and 
software development. This chapter has addressed three of 
the most common software cost estimating methods. The STARS 
program, Chapter VII, addresses a DoD-sponsored software 
initiative which will significantly alter and guide future 


software development efforts for DoD weapon systems. 
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"Software correctness remains the most elusive Sal O2 
computer science. AS.a result, software is the most 
unsafe, the least understood, andthe most expensive 
component of total computer system cost. hee Obeid sta 
cos Peco Weck eG EEeul Cry have shown a_ dramatic 
decrease, especially in the past 15 years, and Bon puter 
hardware capability has improved." Ref. 14 : p. 16] 


A. BACKGROUND 


The preceeding quotation was taken from an article 
authored by the Deputy Under Secretary of Defense (Research 
and Advanced Technology), Dr. Ruth Davis. It expresses the 
concerns shared by many DoD top officials relating tc the 
both cost and safety risks associated with the development 
cr today's computer systems. 

As a percentage of total computer system cost, ai 1S 
generally known that the cost of hardware has decreased 
dramatically over tke the past 15 years while the cost of 
software has steadily increased. Today, software represents 
approximately 90% of the total computer system [{ Ref. 14 ¢: p. 
a8h). There are two Easic reasons to explain the change in 
the cost ratio between hardware and software. These are: 
meet. 15 =: pp. 55 - 56] 

(1) Size: Today's software programs are an order of 
magnitude larger, than they were two decades ago. 
This can be attributed, in part, to the increase in 
size (and Simultaneous decrease in the cost) _of 
onboard memory. Ahn adaptation of Parkinson's law 
suggests that program instructions will continue to 
increase until the limits (and .frequently beyond) 
the available core is fully utilized: 

(ey Complexity: As nature of the applications ene 
automate oe es on Lace oe more sophisticate 
than those applications of yesteryear. Both nmnili- 
tary and ccmmercial Survival strategies are 


Seger el te acral dependent on maintaining the 
ccmpetitive edge inh computer superiority. 


Buy) 


Software has beccme a primary vehicles for solving many 
of the new and changing problems facing the military. In 
Many cases, changes to software is often viewed as an effi- 
cient and expedient way to solve a variety of emerging prob- 
lems or threats facing DoD without having to change the 
existing hardware. Yet the virtues of software are often 
outweighed by its associated problems as described in 
Chapter 21. 

It is not suprising that DoD has identified software as 
the most Significant factor in determining the total cost of 
computer- based systems over their life cycles. Numerous 
studies have been ccnducted which show software quality as 
che cf the most significant factors determining the life 
Gyvcle €COSt. of Sottwame-s This chapter will present many of 
the characteristics cf software quality and the means to 


achieve them. 


Be DEFINING SOFTWARE QUALITY 


Defining quality software is, in itself, atask. fMThere 
are aS many definiticns of what makes Software “goodie. 
there are authors that write about it. Yet these definiticns 
are not mutually exclusive. Each author has his own ideas of 
what the principle characteristics of quality software are, 
and each is right. Defining quality software is as difficult 
as defining the virtues of mankind. Air Force Regulation 
(AFR) 74-1, the "Air Force Quality Assurance Progran," 
broadly and sensibly defines quality as "The composite of 
material attributes including performance." Other, more 
specific, definitions advocated by many of the “gurus §oe 
software engineering will now be discussed. 

Pressman [Ref. 1: p. 148] suggests that good Software 
has three essential gualities: 

==) they SOL EWamc works accordin ue, the _ specified 


requirements-- being as fast, efficient, and as func- 
tional as needed. 
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-- the software is maintainable--it can be diagnosed and 
modified without great difficulty. 


-- the software is more than merely lines of code-~-it 
includes all the supporting documents to ensure that 
the first two qualities are achieveable. 
According to Pressman, good software is based upon _ good 
design, and good design can be gauged by applying a number of 
software engineering measures and heuristics. 

DeMarco {Ref. 8 : pp. 198 - 200] prefers to define soft- 
ware quality as "the absence of spoilage" [Ref. 8 ; p. 200], 
with the term "“spcilage" meaning the amount of effort 
required to find and remove faults introduced during the 
software development process. Eguating this amount of effort 
to its commensurate cost, Demarco provides a formula to 
quantify software guality: [Ref. 8 : p. 200] 

Summation of Defect Diagnosis and Correction Cost 
ee el a a 
Progran Volume 


In which Program Volume is mea 


sured per thousand 
lines of executable code (KLOEC) 


C. CHARACTERISTIC .OF SOFTWARE QUALITY 


One of the most comprehensive and significant works 
written to provide a framework for assessing the issues 
associated with software quality is found in the study 
conducted by Boeha, et al., MmewedmeChaLaceeCriStics of 


SS ee cm me ee 


Quality Software [Ref. 16]. This section will fresent many 
of the highlights rerferted by this study. 

In developing a methodology for the assessing the 
quality cf software froducts, the authors concluded that 
"calculating and understanding the value of a single overall 
metric for software quality may be more trouble than it is 
worth." [Ref. 16 : pe 3-2] A major problem in developing a 
single metric for gauging the quality of software is that 


many of the characteristics of software are in conflict with 
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one and another. For example, Teguirinyg @ high @deq@recaman 
software portability is achieved at the expense ote eona ae. 
efficiency. Coje ccnciseness 1s at oddS with maintain- 
ability, understandability, and so forth. AS Such?) ence ome 
develored a relational set of important Software Chabacueme 
istics which were reasonably exhaustive and non- cverlapge 
ping. This set of characteristics would Serve (towi- ee 
working context for collection and formulation of a Set oF 
candidate metrics used to assess the degree to which the 
software possessed the respective characteristic. Figutemaam 
shows the resulting characteristic set and their hierarchial 
interrelatienshi ps ff Rer-malc. ap. Byes) ie Definitions Gem 


each of the represented characteristics 1s provided in 


aa) mp: ee Se ee ee ee eee eee 
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Figure 4.1 Characteristics Tree 
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appendix A. The characteristics depicted in Figure 4.1 are 
categorized in three hierarchial levels. The higher-level 
structure is oriented toward accommodating various user 
needs and priorities for a software product. For example, 
"As-is" utility is analogous to the "black box" under- 
standing of a system; the user is concerned with only the 
inputs and outputs of the product and need not understand 
the its internal code, nor how to-modify or test it. If the 
Beoduct is going to ke changed by the user, then maintain- 
ability requires that the user be able to understand, 
modify, and test the froduct. 

The lower-level structure depicts those primitive char- 
acteristics, which, although strongly differentiated fron 
each other, “combine into sets of necessary and sufficient 
conditions" [Ref. 16 ; p. 3-25} to define the intermediate- 
level characteristics. The primitive characteristic provide 
the fcundation for fcrmulating the metrics used to quantifi- 
ably measure a software products relative possession of 
those characteristics described in both the high-- and 


intermediate layers. 


D. QUALITY ASSURANCE 


The preceeding section described many of the attributes 
associated with gocd software, as well as their interrela- 
tionships. The purpose of this section is to offer a frame- 
work through which quality software can be achieved through 
planning, specification, and monitoring of quality assurance 
(QA) activities. 

The purpose of software guality assurance, in short, is 
to assure the ultimate quality of the delivered software. A 
formal definition of quality assurance is provided by AFR 


74-1, which defines it as: 
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" A planned and systematic pattern or all actionsmiece 
Sary_ to provide adeguate confidence that material, data 
Supfiies, and services conform to established technical 
requirements and achieve satisfactory performance." 


Another definition fer guality assurance is offered by Pfau 
[Refs 17 : pe 2] who also helped remove some of the sukjec- 


tivity that surrounds the term "quality" Dy Stating: 


"Quality assurance is the name given to the activities 
performed in conjunction with a software product “ge 
uarantee the prodtct meets the specified standards. 
hese activities reduce doubt an risks about eae 

performance of the froduct in the target environment." 


Both cf the above definitions are reflective of the direc- 
tion that QA has taken over the past two decades toward a 
total life-cycle perspective. This evolution of QA has been 
divided into three separate generations [Ref. 18 : pp. 2- 
4}. It 1S important to understand the differences in these 
generaticns in order to avoid the serious pitfalls implic- 


itly and explicitly expressed in the first two. 


First Generation--Test-Oriented QA: This QA generation 
kasically equated QA to software test programs. Tests flans 
and procedures, types of test, and methods of formal verifi- 
cation of performance/design requirements were all essential 
to the testing activity. 

The cbvious and major pitfall of the test-oriented QA 
generation 1s that “you don't test guality into a software 
product." (Ref. 18 : p. 2] Even though testing facet ae 
the discovery of deficiencies, the discovery normally took 
place too late in the development process to allow their 


relatively inexpensive resolutions. 


second Generation=-Development-Orienred aa, Due to the 


inherent failure mechanisms built into test-oriented OA, 


corrective actions were taken by an attempt to make the 


42 


ieweloprMnmeecOntractOrk beSsponsibie for the quality shortcon- 
maps Or the products they produced. Mieteewadssaone by 
Mesiring that the software “delivered under contract fully 
complied with the reguirements of the contract. 

Heiemertral ue toethis OA approach 1s aS limited as the 
Sentracting Officer technical knowledgeable in the tread 
discipline of software enyineeringjg. Contract delivered what 


was specified, nothing more. 


Peeduecet@etatlon=slife=scycle-Oriented QA: ic has 
generation, QA is Luilt into the software from “day cne.!"! 
Mites er.ort 1S properly focused on the early definition 
Prases fOr planning and specifying contractual provisicns 
Reeser ning Software attributes. Figure +.2, {[Ref. 1: p. 25] 


Meristrates the cost impact of introducing changes during 
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Figure 4.2 Cost Impact of Changes 
Meerous phases of the software life-cycle. Pmphasis as 


eee weon the Clear definition of those softwace character- 


istics that were discussed in the first section of this 
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chapter, such as maintainability and portability, which have 
a Significant affect cn the Quality of the product OvVereae 
system's life-cycle. The importance of the life-cycle- 
oriented QA approach and its impact on life-cycle costs 
EOL Owi ng software development and implementation is 


discussed at length in the chapter on software maintenance. 


E. IMPLEMENTATION OF A SOFTWARE QUALITY ASSURANCE PROGRAM 


The preceeding section addressed the definition of soft- 
ware guality assurance, as well as its evolution to the 
present life-cycle-criented perspective which recognizes 
that to achieve the highest quality of software it is neces- 
Sary to include guality checks throughout all phases of the 
software life-cycle. 

This section will discuss the military standards for the 
implementation of software quality assurance (SQA) programs 
in defense contracts. The successful implementation of these 
programs will provide early visibility and managerial 
controls to detect, report, analyze, and correct Soitwame 
deficiencies. Although the focus of this discussion will be 
on defense COntiacts, the methods addressed herein may be 
egually beneficial to in-house development efforts. 

The two most Significant military standards affecting 
the establishment of SQA programs for defense contract are: 
(1) MIL-STD-52779 (A), “Software Quality AsSsirance PrRogmam 
Requirements," providing the basic elements required in an 
acceptable SQA progran, as well as customer evaluation 
criteria, and (2) MIL-STD-1679, “Weapon Systenuso22 7 
Develorment," providing detailed software development stan- 
dards for the entire weapon system software development 
process. Both the software manager and procuring agent 
should cEe familiar with their contents since, togetheng 
these standards provide an effective means to evaluate any 


software development frogram [Ref. 19 : p. 108]. 
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In their article [Ref. 19], Dobbins and Buck discuss 
muye areas Of Control which follow the typical chronology of 
software development. These are: (1) procuring agency evalu- 
ation, (2) design inspection, (3) code inspection, (4) test, 
and (5) library controls. The remainder of this section will 


address each of areas separately. 


1.  Procuring Agency Evaluation 


From both a ccst and effectiveness standpoint, the 
consequences are too important to accept at face value the 
claims that a strong SQA program exists in their organiza- 
tion. There must exist some means to evaluate the potential 
contractor. Major quality items must be addressed as early 
as possitle in the planning process prior to the Request for 
Proposal (RFP) preparation. These quality items should 
include those attributes considered as an integral part of 
the sceftware design, development, test and evaluation, and 
Maintainability issues. Table 4 [Ref. 18 : p. 33] provides a 
humber of factors with which to evaluate bidders! responses 
to the RFP process. 

Cften the program manager and procurement agency 
will have insufficient experience and technical backcsround 
to properly identify essential QA issues needed for inclu- 
Sion inthe RFP nor the means or time to evaluate the 
contractor proposals. In these situations there are alterna- 
tives resources available to evaluate the contractor. The 
first of these is the Defense Contract Administrative 
Service (DCAS). The program manager can hire the services of 
software engineers acquainted current military OA standards. 
Depending on the end-application of the software product, 
there are other government organizations through which 
asSistance can be sought. Other alternatives include hiring 
meme Services of a commercial contractor or consulting firn. 
Regardless of the resource used, a sound means for 


Semeractor evaluation and selection is essential. 
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should be used as the means to collect 
design 
evolved to the more formal 


tion." 


TABLE 4 


Evaluation Factors in Bidder Responses 


FACTORS: 


Completeness 


scope 


Compliance 
Documents 


ye Sree Review 
Boar 

Event Sequences 
Problem 
Reporting 
Action Iten 
Follow-Up 


Past Experien- 
ces/kKesources 


Although MIL-STD-1679 specifies 


Fhase of 


software development, 


The difference between the 


BIDDER EVALUATIGN CHECKS 


Cces bidder's response cover all 
area as requested in the RFP? 


I¢ the scope of the bidder's 
Lesponse COnRSondn tawtcn seme 
Froject's objectives and the level 
of detail anvthe Reew. 


Are all nl eee NS documents 
regarding the software design 
identified? 


Cces the bidder propose to have 
design Review boards? 


Are the reviews of software design 
done in proper seguence? 


Dees the bidder propose to formally 
identify all design problems? 


Dees bidder Prowse. assurance for 
effective follow-up of all action 
items resulting from reviews? 


What experience does the bidder 


have wil QA? Does he have a good 
resource base? 


the process 
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that "walk througie® 


statistics during the 


has 


procedure termed "design inspec- 


two approaches is one "of 


POD gmmiereintent’ fRhert. 19 = pe. 110]. The walkthroughs 
were informal examinations of the software product by its 
authors! technical peers. Little documentation was kept, and 
ho training requirement placed on the participants of the 
walkthrough. 

As with the walkthrough, the design review is a peer 
inspecticn process ferformed by teams that inspect one 
another's work. Unlike the walkthrough, the design review 
is a formal process in which records are kept, and partici- 
pants undergo considerable training requirements. It is 
conducted when the design is completed, prior to the actual 
coding effort. The inspection team is led by a moderator. 
The ideal moderator is not only trained in the technical 
aspects of software engineering, but in the psychological’ 
aspects of software development. 

The moderatcr promulgates reguired inspection 
material to the team in advance of the inspection. Each team 
member reviews the material and records comments before the 
inspection meeting. During the meeting, discussion is 
reserved for major €IQror, i.ee., those errors that will 
prevent the program from functioning properly. Minor errors 
Simply recorded for subseguent correction. Tf more than 5% 
of the program design must be changed due the errors, the 
entire design will be reinspected. Otherwise, the moderator 
will assure that the errors discovered during the design 
inspection are corrected before proceeding into the next 


phase. 


‘For an more in depth discussion of the HS IGNORING GEE 
aspects involved in software development the reader is 
referred to Gerald Weinberg's book the Psychology of 
Computer Programming. ae _ 
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3. Code Inspecticn 


Programming coding may begin only after Successitul compe 
tion of the design inspection. MULL-STD=lo73) © coupe see 
down structured programming and identifles the Specific cease 
constructs allowed. Figure 4.3 [Ref. 20 ; p. 62) ie 


trates the five basic code structures, each having a Sigg 


INPUT 


INPUT 


PROCESS A- 
“PROCESS 8 


QUTPUT 


















TPRHENS EL Se OUTPUT 





SEQUENCE 
INPUT 


PROCESS A 


QUTPUT 


PROCESS 






DO WHILE OUTPUT 


D0 UNTIL case «OUTPUT 


mm ms ccm me mmm a ee cs ee ce eee eee 


Figure 4.3 Basic Code Structures 


data entry and exit. 

Cnce the program is coded and successtully cCcempiieae 
it 1S inspected. The process for inSpecting cole 15 )heame 
identical to that discussed for design. Not only is )theseee 
inspection a method cf discovering coding ertors, ~ouvegu ae 
as importantly it assures that the code e«alheres to the 


approved design. 
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4. Test 


Pertware stesting accclmts for the majority of technical 
efforts expended in software development. Its objectives are 
to uncover software errors, and to provide assurances that 
the software performs its technical and operational 
requirements. 

An effective SQA program must start at the front end 
of software development, with the reguirements specified in 
the RFP, addressing the totality of the testing tc be 
performed. Three measures of software testing relating to 
the RFP include: [Ref. AO59H068 : p. 19] 

(1) The analysis cf software reguirements for test- 
ability. 

MMGeing Getivitics as part “of his Software’ 0. 
Progran. 

(3) The review of test documentation and certification 
cf test results. 

Testing requirements specified in MIL-STD~1679 
require that the system software do more than just just meet 
the specifications. Software must also be subjected toa 
third-partyS "stress test," in which the program is judged 
unsatisfactory if the program execution can be stopped for 
whatever reason. To achieve a degree of software quality 
sufficient enough to fass this type of testing, it is vitu- 
ally essential that the software development program incor- 
porate programs of error detection and prevention well in 


advance of the actual testing period. 


SAs defined in MII-STD-1679, the third party is neither 
the contractor nor the procurement agency. 
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S- iibrary Contrels 


A key element in any SQA program is the software 
library which provides visibility and control o£ | the eee 
ucts documentation and programs. Among the mandatory 
controls stipulated by MIL-S-52779 (A) 1s the cContmegaas 
prevent unauthorized access. Other essential activities ina 
software library are the documentation and program storage, 
and retrieval and change processing. MIL-STD-1679 requires 
a Software Change Control Board (SCCB), which must authorize 
any changes to the controlled library. 


Fe. PARTING COMMENTS 


The underlying goal of software development is to 
deliver quality software. In doing so, it iS Vitale 
examine the characterics of guality, and their interrela- 
tionship, within the context of the user needs and ultimate 
application of the program. To understand the characteris-— 
tics of quality software is to understand the founding prin- 
ciples of software engineering. To produce quality software 
is much more. The implementation of a software quality 
assurance program is the vehicle through which these princi- 
ples are applied andthe goal of software development 
realized. 

The benefits derived from quality software support the 
saying that “quality is free." But more importantly, as wae 
bre addressed in the next chapter, future cost-avoidance 
during the maintenance phase leaves no practical alternative 


to acceptance of only quality software. 


ag 


V. SOFTWARE MAINTENANCE 

This chapter deals with the last phase of the software 
mppecrcycle. Canming [Ref. 21: p. 2] appropriately categor- 
ized software Maintenance as an Wiceberd, Meinitially 
revealing cnly a small portion of maintenance requirements, 
but hiding an enormous potential for future problems’ and 
costs under the surface. With ‘few exceptions, computer 
PeogdramS are always changing in order to correct latent 
errors, add enhancements, and seek performance optimization 
of the software. <A succinct definition for maintenance can 
be given as "that activity which is ccncerned with making 
changes to software for the purpose of i1mproving or 
Meeb&ectang the software." [Ref. 22 : p. 2] Maintainability 
is defined 22 as "a property of software which makes the 
Mainterance activity easy to perform, 1.e., changes tc the 
software are easy to incorporate and do not lead to new 
errors in the software." This chapter will primarily address 
issues of maintainability which by necessity nust be consid- 


ered during all phase of the software life cycle. 


Ae CATEGORIZATION OF MAINTENANCE ACTIVITIES 


Maintenance is much more than just fixing errors that 
escaped detection during the pre-delivery tests and evalua- 
tions. Maintenance has been categorized [Ref. 13: p. 323] 
into four activities that take place after the program is 
released for use. These are corrective maintenance, adaptive 
Maintenance, perfective maintenance, and preventive mainte- 
hance. Each will now be described. 

Corrective maintenance is the process that includes the 


ee ee eee ee se ee es ee sel a Se ee 


diagnosis and correction of latent errors that avoided 
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detection prior to the implementation of the program. It is 
impractical, if not impossible, to exhaustively test complex 
programs in order to guarantee 100% error-free software. 
program as a result of changes to the enviroment in which 
the program must operate. AS an example, it is often quicker 
and less expensive tc modify software rather than the hard- 
waFe in order to modify weapon systems to satisfy new threat 
Situations. 

Perfective maintenance is the process used to accomno- 
date reccaomendations for new capabilities, changes, and 
general enhancements reguested by the user of system 
programmer. 

Preventive maintenance takes place when software 1s 
changed in order to improve its future maintainability. This 
type of maintenance remains a rare practice in software 
engineering. 

Based upon a study of 487 software development organiza- 
tions by Lientz and Swanson, as Summarized in reference 1, 
50% of all maintenance is perfective. Corrective-- and 
adaptive maintenance account for 21% and 25%, trespectively. 


All other types of maintenance account for only 4%. 


B. TANGIBLE AAINTENANCE COST 


Although considered by many software engineers as the 
less glamorous and unexciting phase of software development, 
Mainterance accounts for the majority of the dollars spent 
throughout the software life cycle. The cost of maintenance 
has shewn a dramatic and steady increase over the past two 
decades. As depicted in figure 5.1, one author [Ref. 1: p. 
326] estimates that maintenance cost as a percentage cf the 
total software budget will have grown from 35-40% in 1970 to 
10=S0R 101220. 


ae 





35-40% 40-60% 70-80% | 
Development t Maintenance J 
1970 1980 1990 
l 
Figure 5.1 Maintenance Cost as Percentage of Budget 


Although e@m@pirical data 1S available to account for 
meat SGiLtiware ~tite-cyele cost allocatable to @ainterance, 
Maintenance costs are very difficult to estimate in advance. 
It 1S known, however, that Maintenance costs are often 
dramatically underestimated by both industry and government 
Pwrrng the pre- deployment fhases of SyStem acyuisition. To 
Mmmaottadte this point, Boehm (Ref. 23 3; Dec? ) estimated 
Mitre it took $30 to develop a line of code ({10C), but the 
cost rer LOC skyrocketted to $4000 in the maintenance phase. 
Alithough #4000 per LCC may seem unreasonably high, it is not 
foeetat tO incur such high costs for maintaining mission- 
Critical software in [oD weapon systems. 

Althcugh there is not a set of univerSal factors that 
can Le applied to all software development projects to accu- 
Meeenwestifiate the relative cost of proyram modification in 
Samemmeor 1tS life cycle phases, figure S.2 illustrates the 


exponential rise in maintenance costs in each of the phases. 
fRef. 24 :; p.14]j 
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C. VARIABLES AFFECTING MAINTENANCE COSTS 


As mentioned in the preceding section, 
mate of maintenance costs for a particular program 1s very 
mererredlt.  SOmmervile [Rer. 25 ¢; pp. 198 -199]}] has identi- 
fied five relatively unpredictable factors that contribute 


to the difficulties involved in 


Maintenance. These factors include: 


(1) 


(2) 


(3) 


(4) 


(5) 


=e ae > oe —_ > 


eee ion bein SHOE ed by software is under- 
stcod the better the system requirements can be 
stated. The more definitive the system requirements 
are,. the less erfective Maintenance wiil be 
reguired in the future. 


The application being supported. The better the 


Lifetime of the program. . Toward the end of the 
rogram life, “a déteridration of the program struc- 
Ure GeCcurs due to the multitude of modifications 

iat wares hegany Picat program has gone ene Ou Cie 

Historical evidence suggests that program lifetime 

Peat bt LondevemnMuGh FOnger than Originally esti- 

mated. Hany svotems today daze Still running on 

pregrams that were coded in the early 1960's. 


Dependence of the program on its external environ- 
ment. iiemGCLeCcer sade PrOgram iS tied “t6 Eactors in 
the external environment, the more flexible and 


expandable that program must be to accommodate modi- 
fications due to changes in environment in which it 
operates. 

meats stability. ties Normalivyeeseasier formethe 
Original author of the program to make changes than 
for another rogrammer who must gain an under- 
Standing of the Log een by Sey ng its documenta- 
Eee Pressman [ Ref. 1] uses he term alien code" 
to describe these programs that are extremely diffi- 
cult to understand by those that must maintain them. 


Reasons for alien code include: Cian O (CUERLEH: 
member on the maintenance staff was involved in the 
development of the progran, (2) oor design and 


BOeGcIMNentatton Of the pregramn, andm@(3) nodularity and 
structure sey concepts were not used in the 
development of the program. High. turnover within the 
pee uma profession has made it a rare occurrence 
or the same individuals to develop and maintain a 
PEeegcan throughout its life cycle. 


Hardware stability. Software is designed to be 
compatible with e hardware that will So tere 
Changes _ to the hardware configuration will likely 
resuit 1n requirements for software modification. 


a8, 


an accurate esti- 


deriving cost estimates for 


D. INTANGIBLE MAINTENANCE COSTS 


The direct cost of maintenance, aithough considerable, 
may ke cf secondary concern when compared with the less 
obvious and intangible cost of maintenance. A quote by 
Taniel McCraken [Ref. 1] summarizes one of these intangible 
costs, the development opportunities that are lost due to 


the resources that must be allocated to maintenance CEEorts. 


"Backlogs of new applications and major changes that 
measure i1n io aie ae longer. AS an industry, we 
can't keep up--let alone catch up--with what our users 
Want UY Seo. do. | 


McCraken alludes to what Pressman 1 calla "maintenance- 
bound" software develcpment organization which is no longer 
capable of producing new software because all its resources 
are devoted to the Maintenance of existing software. 


Pressman lists other intangible costs including: 


--Customer dissatisfaction due to the eo! response 
by the software development organization to the user's 
development and maintenance requests 

--Reduction of software qua ese SDrouGgnEwa .oue bY latent 
6€Trors introduced during thé maintenance of software 

-~-Upheaval of devel ent — as personnel and other 
resources are Teale ed™ to work on maintenance tasks 


E. BUILDING MAINTAINABLE SOFTWARE 


Econcmic and efficient support of software is best 
achieved when its maintainability is integrated into the 
total development effcrt from day one. The maintainability 
of software can be quantitatively measured based on the ease 
Ey which it can be understood and changed [Ref. 26 : p. 14]. 

Software understandability is a function of the design 
and documentation. It is easy to understand due to its 


logical and simple structures, and it is supported with 
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documentation that permits an examination of the irplementa- 
tion without losing an understanding for the entire picture. 
Software changeability, on the other hand, is a function of 
the design and implementation. AS an example, implementa- 
tion of modular independence facilitates changes to a 
selected segment by minimizing the degenerative affect on 


other segments. 


® 


1. Structured Methodology 


Building maintainable software is based on the usage 
of a set of software engineering tools and technigues that 
together form a structured methodology for software develop- 
ment. As the authors [Ref. 26] write, the principal elements 


of this structured methodology include: 


"Structured Analysis. A process for developin the 


functional, data, “and interface requirements o the 
software design by constructing a logical model of the 
system process. 

Structured Design. The process of subdividing the soft- 
Ware into hiérarchial nodules ina way tha tends to 
Minimize module independence. 


Structured Programming. The discipline of implementing 


the ccntrol structure of software modules using 4a 
restrictive set of structures. 


Program Design Language (PDL). Language processors that 


are uséd to document Software designS in a structured 
top-down manner." 


Althcugh there are many other disciplines and concerts that 
must also be considered as part of a complete structured 
methodology, such as top-down implementation, structured 
walk-throughs, chief programmers teams, and so forth, these 
Managerial disciplines are used primarily for the software 
development effort. The methodologies underlined above will 
have a visible affect on the software product long after its 
development is completed. A detailed descripticn of the 
characteristics cf each of these elements is provided in 


Appendix C. 
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Although there are many Other disciplines and 
concepts that must also be considered as part of a complete 
structured methodology, such as top-down implementation, 
structured walkthroughs, chief programmers teams, and so 
forth, these managerial disciplines are used primarily for 
the software development effort. The methodologies under- 
lined above will have a visible affect on the software 
product long after its development is completed. A detailed 
description of the characteristics, as provided in [Ref. 26] 
of each of these elements is provided in Appendix B. A mere 


general discussion of each will now be presented. 


2s “Structured: Analysis 


Structured analysis is often considered the starting 
point in the set of structured design technigues. The main 
objective of structured analysis is to build a logical model 
of the desired system. This should be done to the greatest 
extent possible without premature consideration of physical 
implementation. 

In its simplest form, the logical model iS a )pie@eoe 
Tial representation with accompanying narration describing 
the functions, and their interrelationships, of the system 
that they comprise. Fxamples of the some of the most popular 
forms of these graphical representations include process and 
information flowcharts, data flow diagrams, hierarchy chart 
plus input- processing-output chart (HIlPO ehantsoe and 
procedure analysis charts. 

The net result of the this structured analvsis 
process should be a logical model that defines the complete 
system which reflects all facets of the system specification 
and software requirement document. The model should bea 
form of communicaticn easily understood by both technical 
and nontechnical perscnnel, alike. Through the use of this 


model, the system analyst should be to develop system 
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requirements without undue consideration of physical imple- 
Hentatven constraints. On the other hand, nontechnical users 
should readily be able understand how the required functions 


fit together within the context of the whole systen. 


3. Structured Design 


Structured design is the process of of decomposing 
the software design into hierarchial modules in’a manner 
that leads toward independence of modules. Benefits of 
structured design to the development and maintenance of 
software include increased understandability of the system 
and a finimization of the cost inherent in modification. 

Modularity is the key element of structured design. 
It allows for software to be better managed. Large mono- 
lithic (i.e., single medule) programs are often unintellige- 
able to the reader. Modularity is based on a "divide and 
conquer" ccncept, breaking complex problems into comprehen- 
Sible and manageable components. TwO primary measurements 
of modularity are (1) cohesion, and (2) coupling. 

mee COneSiOn is the wieelative Pee Oats Lend th 
[TRef. T : p. 158] of a module. A module is said to be 
cohesive if it .performs a single task within a 
PEogra ml, Feqdulrang  lattile interaction with other 
peegeam code external to its boundaries. In general, 

esign should attempt to realize the highest degree 
cf module cchesion. 

-- Coupling is a measurement of the connectivity among 
other modules. It 1s based on (71) the interface 
complexity between modules (2) the place at which 
entry are reference are made to a module, and (3) the 
type of data that passes across the interface [Ref. 
meee 10 1. — Or. The designer should strive for the 
lowest degree of module coupling. 

Clearly, then, the objective of structured design is 
to Minimize the relationships between modules through the 


Maximization of the functional strength of each. 
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Structured programming is the discipline of itples 
menting module functionality through the use of a limited 
set of programming structures. 

Structure programming uses top-down design by 
starting with the tcpf-level module and decomposing it into 
lower-level modules that it will cali upon. This decomfosi- 
tion process repeated as often as necessary until “tite 
ktottom-level modules are defined. At this bottom level, 
modules make use of EFuilt in operators and functions; they 
do not call on any other module. Each module is separately 
coded using the basic set of program instructions. An objec- 
tive of structured frogramming is to make the design ratch 
the structure of the frogran. 

Any program, regardless of its size and complexity 
can be designed using three basic programming structures. 
The use of this set of programming structures reduces the 
fFrocedural design of the program to a small number of 
predictakle operations, greatly facilitating the development 
and maintenance of software. These structures are illus- 


trated in Figure 4.3. 
5- rogram Design Language 


Program Design Languages (PDL) are language (text) proces. 
sors that are used to document software designs in a struc- 
tured top- down fashion. The goal of a PDL is to replaceron 
EUPPOEt traditional Tebus ewer documentation of program 
design. 

The primary benefits of a PDL are: (1) the documen- 
tation that it produces is normally easier to read and 
understand than flow charts, and (2) the documentation 1S 
always easier to change than are flow charts. Both of these 


advantages are essential during maintenance activities. 
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Fe PARTING COMMENTS 


The maintainability c£ software is inseparable frem_ the 
degree of quality that was built in prior to the maintenance 
phase. Sound software engineering practices, coupled with 
the implementation of managerial controls in maintenance 
activities, offer the key to improved productivity and the 


meonction COStsS associated with maintenance.activities. 
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VI. DOD STANDARDIZATION AND SPECIFICATIONS 


A. INTRODUCTION 


In 1980, it was estimated that DoD spends about $7 
billion a year on software [Ref. 28,3: p. 3]. This amount 
has been steadily increasing as DoD becomes increasingly 
dependent on larger and more complex software products to 
support this generaticn of sophisticated weapon systems. The 
upward-spiralling trend in the cost of DoD software has 
haturally become an area of great concern to officials in 
both military and government. This concern has led (tome 
number of management initiatives in DoD, several of which 
will Fe discussed in this chapter. At the heart of these 
initiatives is the standardization of computer technology 
and software. Standardization 1S seen a means for reducing 
costs associated with the development, operations, and 


support cf DoD computer systems. 


Be ( SEECIEITC. INT TLATIVES 


In her article [Ref. 29 : pp. 37 - 47] Becker describes 
three distinct, but interrelated, initiatives that reflect 
the DoD standardization effort. These initiatives are as 
follows: 

(1) Tne ue development of a Military Computer Family 
yl Gi 


(2) The adoption of Ada as a higher order language (HCi) 
for development of embedded computer software. 


6An instruction set architecture can be described as_ the 
rules and procedures by which hardware executes instructicns 
or computer software. It can also be defined as the erie. 
ture oO a computer that_a programmer must know to write 
time-independent machine language [Ref. 29 : p. 39}. 
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ieee proposed DOL instruction set 
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hi 
m2), 4 le 
The first two of these initiatives will be summarized 
from Becker's article. ines addition, this section will 
address standardization efforts by tier wJOlnNe — Logistic 
Commander's (JLC‘S) panel of Computer Resource Management 


(CRM). 
Pia tary Computer Family 


The distinguishing characteristic in the Military 
Computer Family (MCF) initiative is a common instruction set 
Gaeciitecture. The efforts to develop MCF began in the 
mid-1970's with an intensive review of the Army's misSion- 
critical software. The Army first attempted to obttain an 
existing ISA through a licensing agreement from the commer- 
cial sector. Following an extensive evaluation of this first 
step, the Army concluded a licensing-agreement approach was 
severely limited for a number of reasons: [Ref. 29 : p. 41] 

(1) The adoption of a _commercially~available ISA was 
pereelve as flacing unnecessary technical and 
Metiniistnhat. ve  Bestrictions ~both on the partici- 
pating vendor and the Army. 

(2) The protection and scope of a commercial ISA were 
Pee ere o aS a potential hindrance to the wide usage 

eing considered by the Army. 

(3) Adopting a single firm's ISA was viewed being of 
unfair advantage to one company or ae selected 
segment of the industry, thus greatly restricting 
competition. 

As an alternative, the Army engaged the services of Carnegie 
Mellon University to develop an ISA, which became knecwn as 
"Nebula" ISA and designated by MIL-STD-182A. Nebula has been 
rated as both an effective and advanced ISA. Under a meno- 
randum of agreement. the Army and the Air Force have wecrked 
jointly to develop and control the Nebula progran. 

Using Nebula as the keystone, the Army has engaged 


in a multiphased competitive-procurement process to develop 
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a prototype computer model which will be at the heart of the 
NGEs Although a number of competing companies will be 
involved the the pre-production phases of this develcpment 
effort, only one company will be selected to enter the 
production phase. The number of units acquired during the 
production phase will be based on unit cost as stipulated in 
a requirement agreement that was used aS a criteria in the 
final competition. 

Technological infusion is a major consideration of 
the MCF strategy, ensuring that the MCF has current techncl- 
ogies included in the mission-critical systems that are 
fielded. The Army hopes the the MCF program will result in 
improved survivabilty and logistics, as well as a reduction 


of life cycle costs of the MCF systems. 


2. Ada 


About the same time that the Army began its MCF 
program, the Department of Defense recognized the need for a 
state-of-the-art program for embedded computer applications. 
In the mid-1970's, DoD was spending about $3 billion a year 
on software, with the greatest portion going for embedded 
systems [Ref. 30 : ps. 268}. After concluding that the 
existing programming languages were inadequate for satis- 
fying future software development needs, DoD set up the 
Higher-Order Language Working Group (HOLWG) to investigate 
the development of anew programming language. During the 
four year period, 1975 to 1979, HOLWG published a series of 
mandatory specificaticns for the new language. Each set of 
specifications were more detailed than the preceding set, as 
implied Ly their names; [Ref. 30 : p. 269) 


In 1977, HOLWG studied 26 languages, none of which 
was akle to meet the required specifications. A competitive 
language design effort was initiated 1977. By 1979, the 16 
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TABLE 5 


Ada Specifications 


Strawman 1975 
Woodenman oT 
Tinman 1976 
Tronman 1978 
Steelman 1979 


ee SS ee ee ee) 


— 
| 
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original propositions submitted by industry were reduced to 
one. The winning language was designed by CiIiI-Honey-Eull, 
and was re-named “Ada."7 [Ref. 30 : p. 269] 

The Ada Joint Program Office, under the Deruty Under 
Secretary of Defense (Research and Advanced Technology), is 
responsitle for the management and implementation of all 
Ada-related activities. 

Ada is not without problems and limitations. 
Designed to facilitate a wide range of applications, Ada is 
an extrezely complex and large language. Using context-free 
grammar tokens aS a meaSurement, Adais esStimated at 1609 
tokens leng, Pascal at 500, and Algol-60 at 600. The devel- 
opment of Ada has already been subjected to many of the same 
Criticisms received Fy IBM during their effort to design 
FORTRAN VI. The resulting language, which inccerrfrorated 
features from FORTRAN, Algol, and COBOL was unrecognizable 
aS FORTRAN and was subsequently renamed PL/I. PL/I repre- 
sents the classic "Swiss Army knife" approach to software 
design in which all conceivable features that a user might 
heed are built into a single language. The final product 


being too large andcomplicated for most programmers to 


7Ada 1s a_trademark of the Bona eden of Defense, named 
for Augusta Ada Lovelace, the world's first programmer, and 
daughter of Lord Byron. 
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master [{Ref. 30: p. 182]. AS with PL/I, the size of Ada may 
lead to similar problems as well as inefficiencies in real- 
time application. 

Frovisions and exceptions will have to be made by 
the DoD for existing computer systems whose software is 
written in other languages besides Ada and where conversions 
to the Ada language may not always be possible or feasible. 
However, it will be expected that Ada will be applied where 
possikle, and deviations to this requirement discouraged. 
Fuil isplementation cf Ada is bound to take some time since 
the language, itself, is still ina state of transition and 
because of the huge investment DoD presently has in frograms 
written in other languages. 

It typically takes the better part of a decade for a 
new language to become fully established, but Ada‘'s initial 
acceptance by the commercial sector has been good. Convinced 
that the use of Ada will increase "flexibility and aid in 
the greater utility cf its software packages," [ Ref. 29 : p. 
43] IBM has begun to implement aversion of Ada. Another 
indication of the general acceptance of Ada is the fact that 
the Ada language is in its final stages for consideraticn by 


the American National Standards Institute. 


In April, 1979 the Computer Software Management 
(CSM) subgroup of the Joint Logistic Commanders (JLC) Joint 
Policy Coordinating Group on Computer Resources Management 
(JPCG-CRM), sponsored a workshop at the Naval Postgraduate 
School in Monterey, California--appropriately entiuded@ 
Monterey I. The purpose of the workshop was to review the 
services' software acguisition guidelines. management poli- 
cies and procedures, and Standardization efforts to see if 
there was a basis for the adoption of joint-Service guidance 


in these areaS. Monterey I concluded with the recommendation 
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that the services should adopt common software pclicies, 
development standards, and documentation standards instead 
SewcGontinuing with ¢ach of the service's unigue and often- 
time redundant efforts pertinent to these areas. The advan- 
tages could be attributed to the adoption of joint-services 
standards: 1) econcmies, and 2) the best methods of each 
service could be adopted for use by all [Ref. 31 : p.~192]. 
Other findings of the workshop included: [Ref. 32 : 
eee 1 — 2-9 ) 


(1) No general policy exists for defining,a common soft- 
ware acquisition framework for the joint services. 


(2) A.nugber of diverse regulations and standards exist 
within DoD covering the” various aspects of software 
acquisition and software documentation. 


Peete onol io, OT SO Ce ee Assurance Program 
Requirements," has been widely used since 1974, “and 
MaseDecomeuanl OfLEtClal j7oInt Servaces standard. The 
application of this standard has been met with 
varying degrees of success. Its 2 Se a aie a been 
considered unacceptable due to he impositicn of 
additional schedule and budget requirements. 
Furthermore, DoD plant representatives and DCASR 
Meroonlie! have found 1t most difficult difficult to 
use in the evaluation and monitoring of software 
development contractors. 


(4) Lack of recognized software acceptance criteria a 
PoacinOor DOD StandardyzZation, and a lack of histor- 
ical data upon which to base acceptance criteria and 
procedures. 

Recommendations included the following: 


(1) Develop a general Olicy framework for the jeint 
services to address the entire software life cycle. 


(2) Develop._a unified set of acguisition and development 
Standards for tri-service service application. 


(3) Develop _a_ comprehensive set of data item descrip- 
tions (DID's), subsets which could be used fer and 
software contract. 


(4) Generate a DID for, contractor's software quality 
assurance plan aS a joint service DID. 


(5) Define and develop software acceptance. policy 
rocedures and criteria for the acguisiticn 0 
efense system software. 


The Monterey I workshop concluded with the CSM 


developing a plan of actions and milestones for the 
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implementation of the recommendations listed above, which 
were subsequently aprroved by the JLC's. 

Since receiving the go-ahead from the JLC's, signif- 
icant process has been made in carrying out the irplemeta- 
tion flan ([Ref. 33°35 pp. tee wee The basis for this 
effort was centered around the definition of the software 
development life cycle, with the data item descriptions and 
standards integrated into the appropriate phases of the life 
cycle. Twenty-five kasie Diol s, derined for this furfose, 
replaced a total of over 200 previous ones. This has signif- 
icantly streamlined the documentation requirements required 
for a given acguisiticn. 

The optional practice of conducting a preliminaa 
design review has now been formalized, thus focusing mere 
attention on the requirement definition area of the develop- 
ment effort. This should lessen the problems associated with 
late requirements identification and configuration control. 

A new Software Development Standard (SDS) has been 
written using MIL-STI-1679 (Navy) as one of its basic dog 
ments. The SDS document is at the heart of the development 
effort since it defines the contractor's responsibilities. 
It emphasizes sound scftware engineering practices, such as 
top-dewn design, structured programming, and modulization. 
Other changes to existing standards are being implemented in 
areas such as Configuration Control, Equipment and Computer 
Programs, Specificaticn Practices, and Technical Reviews and 
Audits for Systems, Equipments and Computer Programs. MIwo 
documents have been prepared in the area of Quality 
Assurance: (1) the Software Quality Assurance Measurement 
(SQAM) document, specifying required measurements, and (2) 
The Software Guality Pole. detailing the policies 
governing guality assurance and which will likely replace 
the current Software Cuality Assurance Program Requirements, 
MEL =S 15207 se 
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VII. STARS 


A. OVERVIEW OF STARS 


The scope of the STARS (Software Technology for 
Adaptable ; Reliable Systems) program 1s perhaps’ the 
broadest and farsighted software initiative ever undertaken. 
It addresses almost every socioeconomic, technological, 
Political, and psychological aspect associated with the 
problems of software development and maintenance for major 
military systems. STARS is deliberately structured {Ref. 34: 
pe. 14] to facilitate and encourage the rapid transition of 
new technology into fractice. STARS is intented to te an 
impetus for a cooperative environment among the govern- 
mental, commercial, and academic sectors of U. S. Society in 
which technology transfers will freely occur, and through 
which highly automated and efficient software support envi- 
ronments wili be developed. 

The DoD has a continuing interest in the develcrment of 
computer technology. It is in the best interest of the DoD 
and the country to maintain a front-runner position in 
computer technology. To this end, the DoD has established 
the VHSIC and Ada programs. The VHSIC program (very high 
speed integrated circuit) aims "to gain and maintain a qual- 
itative lead over potential adversaries by providing afford- 
able complex military functions in extremely small, 
ultrareliable packages suitable for operation in severe 
Military environments." [Ref. 34 : p. 16] The Ada progran 
entails the development of a high-order language for mission 
critical ccmputer systems. While both programs have nade 
strides in maintaining American superiority in computer 


technology, a software initiative is being launched to 


aS, 


complement them. STAKES aims to develop the systems and soft- 
ware technigues through which this superiority can be main- 
tained. 

DoD has found that software changes are easier and less 
costly than changes to physical components of militares 
systems. While this can be a major military advantage, the 
needed technology to make these software changes is not 
always available. The software requirements are ahead of the 
systems needed to institute them. Other problems involved 
in the software dilemma besides inadequate technology 
include inappropriate acquisition and management practices 
and a serious shortage of skilled people. Controlling ama 
managing software projects 1S a major concern of DoD. Costs 
for software are becoming the major cost factor on many 
systems projects. These costs must be predicted and 
controlled. The supply of trained professionals is inade- 
quate. Currently the gap "between demand and supply has 
been estimated in terms of 50,000 to 100,000 software 
professionals, and if nothing is done, this gap could become 
860,000 to 1,000,000 software professionals by 1990." 
[Ret< 35 3 Spp. 2 ee 

STARS looks at addressing the technology, management, 
acquisition and personnel proklems in two ways which will 
parallel each other. The long range approach is to "leapfrog 
current technology and completely change the view of the 
software process", as quoted from reference &STAR3.This 
approach is deemed né€cessary since current methodolcecgies do 
not appear to be able to satisfy fully the future reqgquire- 
ments. Opportunities on the horizon which are to be evalu- 
ated include: expert systems, very high level languages, 
functional programming and program generation systems. 
While successful fulfillment of these opportunities will 
enhance the software environment, they will take time to 


develop. The second approach is to "bridge the gap" until 
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Ele Mere <£uluristic Opportunities can be developed. The 
second approach entails an evolutionary strategy of building 
upon the existing systems, improving then, adding tech- 
Nigues, refining models, and training people along tradi- 
tional lines of software development. As stated by Boehm and 
Standish, this approach is necessary to "combat the software 
supply-demand gap". Ey learninj how to manage skillfully the 
large number of variatles involved in software projects and 
integrating the key ccncepts existing in the software envi- 
ronment, managers can utilize their resources needed for 
effective software development. Completeness and integration 
are the key concepts of this second approach. [ Ret. 36 > 
Epa SO —.37 } 


Ea OBJECTIVES 


The primary goal of the STARS program is to "improve 
productivity while achieving greater system reliability and 
meaptability." fRef. 35 <$: op. 56] DoD software in many 
instances is of vital importance in providing life-essential 
mimect ions, such as computerized flight controls. Due to 
this stringent requirement, reliability is of utmost impor- 
tance. The software must be easily adapted to changes in 
mission requirements. A third key element is that of afford- 
ability. As stated e€arlier, cost is an important factor and 
kecoming more sc as more systems are software dependent. 
These three items, reliability, adaptability and afford- 
ability form the backbone behind the goal of STARS. As 
Stated by the initiating task force of STARS, "Wwe need to 
improve the state of practice throughout the DoD community 
so that we can provide development and in-service support 
that is faster, less expensive, and more predictable and 
results in software that is more powerful, reliable, and 


adaptable." [Ref. 35] Based on this goal of an improved 
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software development environment, three basic objectives are 
estaktlisted for STARS: _ 1) expand the level and base of 
expertise in both the government and private sector; 2) 
improve management nethods, appllcation= inderendent- 
technical, and application-specific tools; and 3) increase 
the use of tools by adding incentives, improvements to 
useability and added automation and integration. For each of 
the objectives, a task area has been established with 
specific plans of pursuing the objectives. This pafer will 
discuss the task areas of effectiveness measurements, 


project management and acquisitions. 


"The STARS program will ke carried out Within eee 
context of a variety of on-going and planned activyitiece 
-It will establish a basis or close coordination, 
consistency, and commonalty while pursuing the addi- 
tional work that assures the broad scope and clear focus 
2595°° overall DoD software program."  [ Ret. 3773p 


The program will be instituted in a 7-8 year period. 
Beginning in FY84 with the preparation stage, the following 
three consecutive twe year periods include the consolida- 
tion, enhancement, and transition stages. The consolidation 
stage focuses on putting current technology into practice. 
This includes fully utilizing the management tools, auto- 
mated software tools and implementing the latest procurement 
strategies. The seccnd stage focuses on enhancing the envi- 
ronment established in the first stage. This is an evolu- 
tionary process of refinement and improvement. The fina 
stage will institute a fully funded STARS program. Also in 
this transitional stage any R&D developments which have 
reached fruition can ke transitioned into utilization 1ngea 


software environment. 


UZ 


C. ORGANIZATION 


The -progran is vertically managed under the Under 
Secretary R&D. A jcint Service team under the JUnder 
Peesetatyowill provide the initial planning and coordinating 
of the program. Contractors will assist as required and 
selected as appropriate by various DoD agencies. To aid in 
the government/contractor/academia interface, a tree 
exchange software engineering institute will be established 
to encourage technclogy transfer and thus promote a ccmmon- 
alty of goals and interest. The technology transfer will be 
further enhanced by various DoD agencies't R&D centers 
@enecentrating on their particular area of interest rather 
than attempting to ccver the full spectrum of software engi- 
neering. Also each LoD agency will be assigned resfensi- 
bility of supporting various technology areas. Funding for 
the program is proposed to rise ‘from the $60M level in FY84 
to the $100M level in FY86 (constant FY84 dollars). 


D. EFFECTIVE MEASUREMENTS 


Measurement of key elements ina system allow one to 
understanding the system process and therefore control the 
process [Ref. 38 : pre 47 - 53]. (epnirasaning Control, and 
predicting outcomes in software development projects isa 
major advance in software technology. Practical benefits of 
being able to achieve effective measurements include: 1) 
provides a descripticn of the software environment; 2) 
allows possible prediction of project parameters such as 
cost, delivery time, constraints, and quality; 3) permitting 
the expression of requirements and goals quantitatively; 4) 
ability to track progress and provide feedback; and 5) 
providing a means of analyzing costs and benefits. While 
these benefits are great, Csratmiings=tne “abliity to “have 


reliable measurements is a task unto itself. 
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Two areas needing effective measurement are software 
performance and user performance. Software performance 
becomes more important as software plays a larger role in 
the overall system. Software systems must be able to inter- 
face and effectively synchronize to function properly. 
Performance of users haS an impact onthe cost and time 
required to.produce systems. Studies have shown that devel- 


oping reliable models to predict such performance is near to 


impossible. 
STARS intends to institute ao eine: Olen methcd of 
apprceaching the measurement task. In keeping with the 


overall goal of STARS, an environment conducive of model and 
metric development will be evolved. In general terms, the 
development and refinement of existing models will continue. 
More data will be gathered andthe iterative process of 
hypothesis testing will continue. There will be a widespread 
emphasis on using measurement tools and models. Manual as 
well as automated tocls will be made standard as much as 
possible. With an increasing data base, baselines will be 
defined and maintained. These baselines will include size, 
effort, reliability, and the use of methods and tools] am 
in all the benefits of the measurements will be to allow the 
assessing of methods and tools in order to get the most 


product from the least amount of resource expenditure. 
E. PROJECT MANAGEMENT 


"The primary objectives of the project management task 
area are: $1) enhance the buyer mahager's capability in 
early Boece planning; 2) provide a better means of 
COlmunIcating sand coordinating between and within buyer 
and prcducer organiza tions, ) furnishing tools toga 
Managers in identifying and correcting problems Lefore 
they affect schedule or functional capability; and 4) 
increase the availability of software engineers educated 
a4y°7° principles cf project management. (Ref. 39 2c 
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Most software system development projects involve the LoD 
and a contractor with the DoD component being the buyer and 
Bren GOntcLactor being the producer. Eariy projecteplanning 
performed by DoD project managers is often lacking. Many 
projects reach the award stage before proper planning has 
taken place in the areas of mission analysis, requirements 
Gefiniticn, scheduling and cost identification. This causes 
problems of unspecified work statements and misguidance of 
contractors in the early contract period. The bottom line is 
Brat ~COr planning ccstS money. STARS intends to overcome 
memes threugh general guidelines in the pre-award contract 
phase. 

The second objective deals with communication between 
the SenubacroOr ald the government and the overall 
contracting process. Communications are intended tc be 
improved through better documentation and the building of 
closer working relations between the contractor and the 
government. The ccntracting process will be addressed 
through the establishment of a software acquisition fanel. 
The panel, made uf of various Service representatives 
including STARS and input from industry, will reccmmend 
appropriate acquisiticn policies, contract incentive mecha- 
nisms, and make reccmmendation and promote changes to the 
software systems acquisition frocess. 

The third objective is to equip the manager with a stan- 
dard "tool kit" consisting of management tools which will 
allow identification of problems before they can impact 
greatly on the project. This tool kit should also be avail- 
able to the contractor so that communication will be along 
the same lines. Examples of tools are: data base managers, 
word processing, telécommunications, graphics, spreadsheets, 
schedule generator, cost estimation and general reporting 
systems. The aim of the tools is to automate the tracking of 


the project. 


fis: 


The final objective is that of educating the prcjeas 
managers into the profer management perspective. This calls 
for the development of standard job descriptions followed by 
training in the areas of project management. This objective 
is important since most individuals involved in project 
Management of software systems were or are software profes- 
Sional and not management professionals. 


® 


Fe IMPRCVING PERSONNEL RESOURCES 


Overall, the demand for software is increasing at 12% 
per year, while the supply of software-producing personnel 
1S increasing at an annual rate of only 4 percent. 
(Ref. 40: p. 31] If this trend continues, the Shortagemes 
software-producing personnel will increase tenfold te an 
estimated shortage just under one million software profes- 
Sionals Ly the year 1990. Each of the services have already 
reported shortages cf qualified software personnel and 
predict that these shortages will become critical by the 
late 1980's. [Ref. 35: p. 53] Another area of concern is 
maintaining the skill levels of present software personnel 
abreast of the skill level demanded by rapidly changing 
technology. 

The task objective to improve personnel resources 1S 
based on two fundamental premises: (1) increasing the level 
of expertise, and (2) expanding the base of expertise avail- 
able to DoD. The strategy and major subtasks for achieving 
this objective is presented in detail in the article by 
Orglesby and Urban [Ref. 41 3: pp. 65 - 70] and will now be 


highlighted. 
1. Kev Populaticn Assessment 


This major suktask is designed to assess the human 


resource issues of the availability, the utilization and the 
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Mricewseqlinements OL SoLtware-related sxilis. Only through 
these assessments can skill requirements for software- 
related skills be determined. Quantitative measurements 
based on educational units and/or task period performance 
would then be used to for qualification and classification 
Or employment and career development of software 


professicnals. 
2. Career Structures and Incentives 


Cnce the key fopulation assessment is completed and 
skill reguirements kncwn, career structures (career ladders) 
can be developed and fut into place for each of the occupa- 


tional subspecialties within the software-development field. 


3. Exchange Programs 


This subtask is structured to increase the number of 
software personnel exchanges for prescribed periods amcng 
government, industry, and academia. Regulations are already 
in place permitting personnel exchanges between the services 
and between DoD and state organizations. These established 
exchange programs are to be better publicized and supported. 
Exchange programs will be initiated with industry, DoD and 
academia. These programs offer an excellent medium for tech- 
nology transfer, training, and a better understanding cf the 
problems associated with a counterpart community, be it 


misiae Or outside of DoD. 
oe ther Educational Subtasks 


Other educaticnal subtasks contemplated under STARS 
to improve human resources include: (1) academic programs 
that will encourage the development or enlargement cf soft- 
ware engineering programs in colleges and universities, (2) 
training programs utilizing governmental or nongovernmental 


programs to advance the educational technology in software 


1 


engineering with efforts oriented toward Ada technology, and 
(3) learning aids that focus on automated instructional 


systems and knowledge-based tutorial systems. 


G. IMFROVING PROCESSING TECHNOLOGY 


The second approach taken by STARS to help develop a 
software support environment is through the improvement of 
processing technologies. Processing technology includes the 
"technigues, methods, practices, and tools supporting soft- 
ware over its complete life cycle". [ Refl 3702 sone 

One way in which this objective can be met is through 
imprcved application-independent technical tools. These are 
tools that support projects of all types, regardless of 
app liecatiron. Examples of application-independent tocls 
include operating systems, linkers, loaders, compilers, and 
programming languages. An example of the latter is Ada, 
which is the cornerstone of current efforts directed toward 
the development of the Ada Programming Support Envircnnment 
(APSE). The long-term objective of APSE is to provide a 
common high-order language through which programming support 
environment tools can be interfaced. However, for the short- 
term it is necessary for APSE to accommodate the multilin- 
gual inheritance of DoD's diverse, programming-support tool 
inventory. [Ref. 42 : p. 15] 

A second way in which this objective can be achieved is 
through improved application-dependent technical methods and 
tools. [Ref. 34] Examples of this category of tools include 
Very High Level Languages (VHLL), libraries, test drivers, 
and simulators. 

Mid-- to long-term objectives of these application- 
specific task areas involve the use of emerging technology, 
such as VHLLs, Knowledge-based systems, and program genera- 


tors. The short-- to near-term objectives (next seven years) 
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Of — this task are are centered around the software 
"reusability" problem in which software for each new svstem 
has been developed in total, from- the-ground-up, as though 
it is the first and last system of its kind. Future efforts 
will be directed tcward the development of Ada-kased 
reusable software. Reusable software is hardly a new idea, 
but past attempts to create sets of reusable software have 
Mmatred for lack of guality control. To overcome similar 
problems, DOD's software must be developed with the 
Heomlowing Characteristics; [Ref. 37 : p. 79] 
-- precise statements and validations of module func- 
icns and interfaces. 


-- generalized performance functions to increase scope 
Ge application. 


peause Of high Boe aa standards and widely-accepted 
programming methcdologies. 

-- robust behavior. Not only must software be reusable, 
reusable software must also be accessible by software 
developers. Techniques for cataloging and ware- 
housing reusable software must also be implemented. 
Curren data tase management technigues for_ the 


query, . management, and retrieval are considered 
meplLoOPEeate £Or this application. [Ref. 43 : p. 18) 


He INCREASING USE OF PROCESSING TECHNOLOGIES 


Improved processing technology for software develcrment 
can only make a difference if people use this improved tech- 
MOLOGY. Another objective of the STARS program is to 
increase the appropriate use of these technologies. Two of 
the subtask area in supporting this objective are (1) 


improve business practices, and (2) improve tool usability. 


1. Improve Business Practices 


This subtask is aimed at changing current DoD regu- 
lations in order to facilitate the acquisition of software. 
Another goal of this subtask is to utilize financial incen- 


tive schemes to encourage capital investment by industry 


LS, 


directed at the coordinated pursuit of new technology devel- 


opment. 


2. improve Tool’ Usability 


This subtask focuses on improving the interaction 
retween computer-based systems and the users or developers 
cf software. in her article, [Ref. 19] Elizabeth Kruesi 
lists three basic objectives in this area: , 

-- te expand the technology base By Supporting the continued 
development of knowledge, methods, and tools for incorfo- 
rating human factcr cofcerns into system development, 


~~ to. expand the experience base Oee the application of 
this technology to actual development projects, an 


~- to ensure effective human factor engineering of autcmated 
gpecial needs oF softvare professiona, 
Although Kruesi suggests many techniques and methods 
for improving tool usability (such as defining user inter- 
face goals, early user testing, predictive tools for inter- 
face design, and following proven interface-design 
guidelines), she sees imperical testing as the one human 
engineering method cffering the most promise. Although 
noticeably lacking in the past development of tools and 
envirenments for software personnel, imperical testing is 
viewed by Kruesi as an especially "rich source of ideas for 
user interfaces, particularly in the design of advanced 
software environments such as Smalltalk and Interlisp...." 
Although one of the benefits that will be realized 
from the improvement cf tool usability is increased preduc- 


tivity, the primary Fenefit may very well be in the avoid- 


ance of human error in the design, development, and 
Maintenance of life-dependent and mission-critical Spam 
systems. 
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ioe CNCLUSTON 


This chapter has presented many of the managerial and 
technologically-oriented objectives that DoD has inccrfo- 
rated under the STARS program. This software incentive is 
enormously broad in its scope, including all major sectors 
of society in both fresent and future efforts to keep this 
country at the forefront of software technology. Although 
still in its infancy, STARS has defined many existing soft- 
ware prceblems and has established both evolutionary and 
revoluticnary strategies to minimize these problems in the 
future. The conceptual foundation of STARS is sound and 
promises to improve future software development should the 
program receive the financial support that it deserves. 

STARS 1S an aggresSive approach to a well defined set of 
problems. The key to the success of STARS, as is true of any 
government initiative, is the widespread acceptance of the 
@encerpts surrounding it. The key element driving STARS is 
that of standardization as supported through comnmonalty of 
methodology, uniform metrics and baselines. The software 
institute calls for a sharing of information and the ever- 


increasing technology transfer. 
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VIII. CONCLUSIONS AND RECOMMENDATIONS 

The gains made in software engineering over the past two 
decades have been significant, yet software projects fail- 
ures continue with alarming regularity as both the- size and 
complexity of computer-based systems continue to grow. 

There 1S no shortage of proposals to confront the frob- 
lems that plague the development and acquisition cf soft- 
ware. Yet, the very nature of software continues to defy its 
guantitative analysis resulting in obscured visibility and 
ineffective controls in the development and maintenance 
process. 

Although great strides have taken place in the fcrmaula- 
tion of software metrics as management information tools and 
as a medium to provide feedback to software engineers, 
attempts to devise metrics to guantify software guality have 
remained elusive. Software quality assurance programs, such 
as described in MIL-STD-52779(A), provide a planned and 
systematic approach for building quality into Softwares 
Successful implementation of these programs have given 
credence to the saying the "quality is free," in the long 
run primarily through cost savings inherent in truely nain- 
tainatle software. 

Soitware maintenance 1s the neglected phase in the soft- 
ware life-cycle. Maintenance accounts for well-over half of 
all resources expended on software throughout its life. The 
trend in the amount of efforts needed to maintain software 
is increasing at a dramatic rates, consuming resources that 
were once reserved for developmental efforts. Yet, project 
management does not cften give sufficient consideration to 
building maintainability into software as an indisfensikle 


criteria inthe design process. Various technical and 
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Managerial approaches can be implemented within the mainte- 
nance activity with Dinimum upheaval, but the most influen- 
cial factors leading to the maintainability of software 
cccur during the phasesS prior to the maintenace phase. 

Chapter VI focuseS on various high-level efforts 
directed at the standardization of computer technology and 
software within DoD. If the appropriate selection of tools, 
methods, and methodologies advocated by current software 
literature and directives were to be put into practice, 
better software would be realized in DoD. In order tec 
assure the success of this undertaking, it has been 
suggested by numerous authors that the project manager 
Should be provided with sufficient technical background. 
This approach is the likeliest to assure failure. Today, the 
framework in which DoD software is acguired and developed is 
both disjointed and rerplexing. The myriad of instructions 
and guidelines offer platform of confusion not resoluticn. 
Significant progress has been made by groups such as the JCL 
in attempting to standardize, through joint-Sservice instruc- 
tions, many of the aspects affecting the acguisition and 
Maintenance of software. Much remains to be done. 

The immaturity of the software engineering discipline 
has been too often been pointed to as the primary culprit of 
software failure. Software engineering must never mature; it 
must continue to evclve at a pace set by advances in our 
technological society. Software engineering is but one of 
the factcrs contributing to the delivery of guality soft- 
ware. Standardization is the key in reversing the trend of 
the delivery of overkudget, overschedule, inferior software. 

The management and development of software today is like 
trying to understand a United Nations assembly without 
interpreters. Today there are far more programming 
languages than there are different languages at the United 


Nations, with revisions bastardizing the integrity of its 
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parent programming language as dialects bastardize their 
mother language. Yet programming languages is just one 
aspect of the total standardization effort that must take 
place in DoD. The best of today's management systems can be 
consolidated into a single, joint-service system understood 
by management personnel in both DoD and industry. The adop- 
tion of any one set of tools, methods, and methodologies for 
the development and acguisition of software is far better 
than attempting to live by all of the sets available. 
Benefits derived through Standardization should be 
exploited to their fullest. The broadest and most farsighted 
of these efforts is the STARS program, which addresses nost 
aspects that define the total software life-cyake 


envirenment. 
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GLOSSARY OF SOFTWARE QUALITY ATTRIBUTES 


Definitions provided in this appendix are derived from 
fRef. 18 ; Appendix Bj and [Ref. 16 ;: pp. 3-4 - 3-24]. 


NOGEoSLBILITY: Code possesses the attribute accessi- 
Prelit y aS the degree that it facilitates selective use of 
mis parts. 


_.. ACCOUNTABILITY: Code possesses th 
billity to the degree that it degree t 
measured. 


e attribute accounta- 
hat its usage can be 


ACCURACY; Code fossesses the, attribute accuracy tc,the 
degree that its outputs are sufficiently precise to satisfy 
their intended use. 


_.. AUGMENTABILITY: Code possesses the attribute augment- 
omeeieye tO the degree that it Can easily accommodate expan- 
Sion in component ccmputational functions of data storage 
requirements. 


COMMUNICATIVENESS: Code possesses the attribute communi- 
cativeness to the degree tha 2c facwilitates the Specifica-— 
tion of inputs and provides outputs whose form and content 
are easy to assimilate. 


CCMPLETENESS: Cede possesses the attribute completeness 
to the degree that its parts are present and each part is 
fully developed. 


This implies that external references are available and 
required functions are coded and present as designed. 


CCNCISENESS: Code possesses the attribute conciseness to 
the degree that excessive information is not present. 


CCNSISTENCY: Code possesses the attribute consistency to 
the degree that it. ccntains uniform notation, . pe ee Ne 
an = EP within itself, and external consistency to é 
degree that the content is traceable to the requirement. 

DEVISE EFFICIENCY: Code possesses this attribute to the 
@egree that it that the operation, function, or instructions 
provided by the ccde are erformed without waste of 
resources with respect to that device. 


DEVICE-INDEPENDENCEs Code possesses this attribute to 
the degree that it can be executed on computer hardware 
configurations other than the current one. 


EFFICIENCY: Code possesses this attribute to the degree 
that it fulfills its furpose without waste of resources. 


HUMAN ENGINEERING; Code possesses this attribute to the 


degree that it fulfills its ge SO without wating the 
user's time and energy, or degrading their morale. 
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LEGIBILITY: Code possesses this attribute to the degree 
that it is easily discerned by reading the code. 


MAINTAINABILITY:. Code possesses this attribute to the 
degree that it facilitates updating to satisfy new require- 
ments or to correct deficiencies. 


MCDIFIABILITY: Code possesses thiS attribute Comper 
degree that it facilitates the incorporation of ne ae 
once the nature of tke desired change has been determined. 


PORTASIDETY -aeode Mot asi oe this attribute to the degree 
that it can be operated easily and well on computer configu- 
raticns cther than 1tS currenteore: 


RFLIABILITY: Code possesses this attribute to_ the degree 
that it can be expected to perform its intended functicns 
satisfactory. 


ROBUSTNESS: Code pose eee this attribute to the degree 
that it can continue to perform despite sone violatvcnmas 
the assumptions in its specifications. 


SELF-CCNTAINEDNESS: Code possesses this attribute tc the 
degree that it performs all its explicit and implicit func- 
ClONS Within teseits 


SELF-DESCRIPTIVENESS: Code possesses this attribute to 
the degree that it contains enough information for a reader 


to determine and verify its “objectives? assumptions, 
COR Eee ates inputs, outputs, components, and revision 
status. 


STRUGIUEREONESS: -) Code ossesses this attribute to the 
degree that it contains a definite pattern of organization 
of its interdependent parts. 


TESTABILITY; Code possesses this attribute to the degree 
that. “Le saci eates soienc establishment of verification 
criteria and supports evaluation of its performance. 


UNDERSTANDABILITY: Code_possesses this attribute to the 
degree that its purpose is clear to its inspector. 


USABILITY: Code fossesses this attribute to the degree 
that it is reliable, efficient, and human-engineered. 
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APPENDIX B 


HADOLEAD AND MECABE'S SOFTWARE METRICS 


As generally addressed in Chapter III, this appendix 
will present quantifiable measurements of various software 
charateristics using Ltoth Halstead's Software Science Theory 


and McCake's Complexity Measure. 


Fem HALSTEAD'S SOFTWARE SCIENCE 


Halstead begins with four basic metrics: 
1. ni--the number of unigue or distict operators on 


the progran. 


2. n2--the number of unique or distinct operands in 


the progran. 


3. Ni--the total usage of all the operators in the 


program. 


4. N2--the total usage of all the operands in the 


program. 


Tables 7 and 8 show the resulting counts of the oferators 


and operands used the algorithm used in table 6. [fRef. 10 
mop. 3 =~ 18} 


The vocabulary (n) of a module is described as the sun 


of its unique operatcrs and operands: 


N= nt + 72 
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ess 
| TABLE 6 
| A Sorting Subroutine | 
SUPROUTINE SOK? Visa 
DIRE NS EON iy | 
189 4h Sih ivs \ REI Uee | 
DO 20 et. | 
DO 10 og = "dem | 
IF (0 (i) (acer X(J)) GO Toe | 
| SAVE = 3} 
x 13} = X(J 
X (J) = SAVE i 
10 Ce NGENWE { 
20 CCNTINUE 
RETURN | 
| EN | 
ae se a a nS SS a a tS 
| | 
TR Bee | 
| Operator Count | 
| 
Operators Count | 
1. End of statement 7 | 
aS Array subscript eZ 
4. IF () 2 | 
5.5.00 2 
oe J | 
7. End of Frogran 1 
a e dee 1 | 
9. @ @ 1 
10. GO TO 1 | 
ni = 10 N1 = 28 | 


|e ee 


Similarly, the length (N) of a module is defined as the 


sum of all operators and operands used in the module: 


N= N1 + N2 
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| 
TA BIS 
Operand Count | 


Operands Count 

I X 6 | 
ee Aa 5 
Se J a) 
4. N 2 | 
Dis 2 2 | 
6. SAVE Z 

a 1 1 
n2 = 7 N2 = 22 | 


Se a a A ce wee nee eee al 


idength (NH); 
NH = ntlog2n1 + n2log2nz2 


The estimated length equation (NH) has proven an accep- 
table estimator for the length, N, of a nodule. The useful- 
ness of NH seems to be somewhat sensitive to actual prcegram 
length. Test have shown that this formula works best for 
programs if N is in the range between 2000 and 4000. In this 
lignt, errors can minimized by breaking down modules to 
where they are withir these parameters [Ref. 11]. Halstead 
attributes this finding to the presence of "impurities!" in 
the freogram requiring optimization. 

When compared against the actual length of the above 
listed algorithm one finds that N = 50 and NH = 52.9, a 
difference of less than 5.8% in this particular case. 

Additional metrics were defined by Halstead using the 
terms already presented. Of interest is another measure of 


program size called volume, (V), which is measured in bits: 


V = N x log2n 
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Volume may also te interpreted as the number of mental 
comparisons needed tc write a program of length N, assuming 
a binary search method is used to select a member of the 
vocabulary size n. The most sSsuccint form in which an algo- 
rithm can be expressed reguires prior existence cf a 
language in which the required operation has already been 
defined cr implemented. In such a case, the implementation 
of that algorithm would require no more than naming of oper- 
ands for its arguments and its resultants (eg. SORT (Qe 
These algorithms are considered minimal in size and are said 


to have the potential volume V#: 


Ve = (2 + n2) x Log2 (2.4m 


(where n2* is the different input and ouput parameters) 


Any program with volume V is considered to be implemented at 


the program level, L, defined as: 
L = V¥/V 


Notice that for the most succint version of any algorithao, 
the resultant 1s 1. As the unigue operators (n1) increase 
and the reuse of oferands  (N2) increases the resultant 
approaches) 0: The term "difficulty" is derived from the 
logic that as the volume of a program increases, the program 
level (L) decreases and the difficulty increases. Thus, 


difficulty (D) is the inverse of the program level 


D= 1/1 


Since the volume (V) is the number of mental comrfari- 
sons, and the difficulty (D) is the measure of the average 
elementary mental discrimination required for each mental 
comparisen, then by cecmbining the formulas for L and D, the 
total number of elementary mental discriminations, effort, 


(E), required to generate a program can be derived from 


E = V/L 
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The advantage of Holstead's measurement of effort (EF) is 
agesiqgniticant break from traditional use of LOC's. fhe use 
of LcCts required the collection of data and regression 
analysis. Only the number and use of operators and operands 
are needed to derive measurements for effort, thus over- 
coming the forementioned difficulties of using LOC's method- 
Slogvy. Another advantage of using E .1s that it is a strong 
indicator of the ccmprehensiblity of a program and its 
Peewmensity for errors {Ref. 10 ; p.~ 16] and [Ref. 11 : p. 
34}. 

Eeecringetne formula for L further, it can tke noted 
that as the potential volume (V*) increases, the progran 
level (1) decreases proportionately. Consequently, the 
product L times V* remains constant for any one language. 
This product, the language level, which is denoted as 
LAMBDA, is derived by the following formula 


LAMBDA = L x VY 


The last of Holstead's formulas to be discussed is used 
to measure the programming time (TH) for a prograr in 
seconds. Halstead adopted the concept introduced by John 
Stroud, a psychologist, who defined a "moment" as the tine 
required by a human brain to perform the most elementary 
discrimination. These "moments" according to Stroud, occur 
Saad «6fate Of 5S to Slightly less than 20 per second. 
Halstead then deterzined that the programming time of a 
program could mathematically be defined as 


c=) BY 


(where S = the "Stroud" number = 18) 


Halstead reason for selecting "18" for the value of the 
Stroud number remains a myStery, BiteheetatS his formula 
hicely andis likely to remained unchallenged until the 
disciplines of cognitive psychology and software science 
merge. 
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B. ACCABES*S COAPLE SID ensues 


A pregram graph is used to represent control flow, as 


—— me ee ee ee ee ee es eee ee ee ee ec ey ee eee eee 






Entry Node 


Exit Node 


mm mmm mmm meme gm mee me ne ee ee ee ee eee ee ee ee eee 


Figure ou. Control Flow Graph Complexity 


illustrated ane Figure B11 [ Ret. cla PP. 308 ~-3200 0a 
circles represent processiny tasks, which can be one or more 
SOugCcEe Code statements. Arrows depict conta flow 
(branching) between frocesses. Thus, in Figure B.1 , process 
"a" may be followed ky process "b," "c, "Mor “dy depen aem 
on which condition was satisfied in process "a." The ccntrol 
flows depicted by arrews yoing from process "e" to processes 
"Db" and "ct represent "backward" Lranching. "Seq 1ons see ee 
descriked as the enclosed areas on the plain of the graph 
represented by R1 through RS in Fijuresy@ega eee These 
regions represent the bounded areas within the prcgraam 


graph, as well as the unbounded area outside of the graph. 


eZ 


McCabe uses a software complexity measure that is rased 
on what he terms the "cyclomatic complexity" of a prcegram 
graph for a module. One approach that can be applied in 
determining the cyclomatic complexity, ViGl, 1s by calcu- 
lating the numbers of regions in a planar graph. In Figure 
B.1 , for instance, V(G) is equal to 5. Another method is 


through the formula 
V(G) = e- n+ 2p 


in which "e" is equal to the number of "edges," 1.e. number 
of arrows, Vatiis (equal to the number of “vertices,” or 
processes, and "p" as equal to the number of connected 


components. In Figure B.1, the values of these elements 


are; 
N= sotay, DD, 7G, dad, ¢€, £) 
ef-— 9 Wide tol ~d tO 1c, a to d,.b to.e, ¢€ to 
i eCeeCO apm LOmec, Cato foc tof } 
. a 
By inserting these values into the above formula, the 


resulting cyclomatic complexity metric, V(G) is again equal 
five. McCabe also contends that the V(G) measure can provide 
a quantitative indication of the maximum size, testing 
Peer ieulty, and reliabilty of a module. Through empirical 
investigation, he has found that a cyclomatic complexity 
measurement of 10 tc be a practical upper limit for module 
size. Exceeding this upper limit makes it increasingly 


difficult to adequately test a module. 
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STRUCTURED METHODOLOGIES 


As discussed in Chapter V, the design of maintainable 
software is based on the application of a set of engineering 


princifles and practices that include: 


-~- Structured analysis 
-- Structured design 
== Structured progranming 


-- Frogram design language 


This appendix presents the detailed characteristics cf these 


elements as provided in [Ref. 26 : Appendix A]. 


A. STRUCTURED ANALYSIS 


When applied to software development, the characteris- 
tics of structured analysis and the logical model areas 
follews: 

-- A system_is described by the systematic decomposition 
Of rroad system functions “into Subunctions of 
Frogressively finer detail. 


== PaGh vy. nou lmono wana subfunction is defined by 
describing 


Its inputs and outputs 
Processing activities and requirements 
Nontransient data stored by the function 
-- Functions and sutfunctions are analyzed to access 
The support of functions by hardware and software 


Algorithm and computational requirements 
(Function, precision, Edange, inane oe 


The need fcr performance and tradeoff studies 
-- Stored and interface data are analyzed to access 
Access requirements 


Structured, format, and storage requirements. 
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-- Functions and data are ahalyzed to evaluate the 
requirements for execution and support software. 

The net result of the this structured analysis process 
should be a logical model that defines the complete systen 
which reflects all facets of the system specification and 
software requirement document. The model should be a form of 
communication easily understood by both technical and 
nontechnical personnel, alike. Through the use of this 
model, the system analyst should be to develop system 
requirements without undue consideration of physical imple- 
mentation constraints. On the other hand, nontechnical users 
should readily be able understand how the required functions 


fit together within the context of the whole systen. 


B. STRUCTURED DESIGN 


Structured design 1s the process of subdividing the 
software design into hierarchical in a manner that tends to 
Raximize module independence. Benefits provided to _ the 
development and maintenance include increased understan- 
dility of the system and a Minimization of the expenses 
associated alteration of the software. 

The structured design approach has the following charac- 
teristics: 


-- Hierarchicial functional tree charts are develored 
shoein a series of boxes representing descending 
Wevels of functions and subfunctions. These charts 
depict the functionality in the same sense that a 
Sas cae work. Ereakdown structure (WBS) depicts 

lefarchies of work to be performed by a contractor. 


-- Modularity of ccmponents is emphasized. A key charac- 
teristic of modularity 1S a maximum independence of 
One component from others. This independence allcws 
for a concentratior on definition of inputs, outputs 
and processing. of each component. It also facili- 
pee coe +o8 Clarity, which facilitates future modi- 

ie ce Ons . 


-- At each level of component design, strong emphasis is 
Flaced on defining pape Se outputs, and processing of 
each component. This emphasis represents ae key 
characteristic of both structure design an 
analysis. 
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C. STRUCTURED PROGRAEMING 


Structured programming requires that a programmer use 
only a limited set of three basic program structures. 


These are depicted in Figure 4.3, and are as follows: 


-- Seguence of two or more operations 


-- Conditional branch to one of two operations and 
Eeturn (LF “a LTHEN beees se 


~- Repetition of an operation (DO WHILE) 


Any program, regardless of its complexity, can be imple- 
mented using these basic program structures. The use of only 
these structures limit the procedural design of a program to 
a Small amount of fredictable operations. It should be 
noted, however, that use of only these three structure may 
lead to inefficiencies in situations such as when an escape 
from a set of nested conditions or loops is needed. In situ- 
ations such as these, the designer if left with the options 
of re-design to avoid these conditions or allow for devia- 
tion from these basic structures in a controlled manner. 

Two extensions to the basic structures, also illustrated 
in Figure 4.2, are the DO UNTIL and CASE structures. These 
are special cases of the other structure which improve both 
redability and source code without degrading programming 


Structure. 


D. PROGRAM DESIGN LANGUAGE 


A program design language is a basically a test 
processor that is used to document a structured design. It 


has the following two characteristics: 


= roduces an English-like representation of comfpo- 
fet s of code that are easy to read and comprehend. 


-- It is structured in the sense that it uses structured 
programming constructs to Show nested logic. 
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